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DIFFUSION  APPROXIMATIONS 

Peter  W.  Glynn 

Department  of  Operations  Research 
Stanford  University 
Stanford,  CA  94305-4022 


1.  Introduction 

In  this  chapter,  we  shall  give  an  overview  of  some  of  the  basic  applications  of  the  theory 
of  diffusion  approximations  to  operations  research.  A  diffusion  approximation  is  a  tech¬ 
nique  in  which  a  complicated  and  analytically  intractable  stochastic  process  is  replaced  by 
an  appropriate  diffusion  process.  A  diffusion  process  is  a  (strong)  Markov  process  having 
continuous  sample  paths.  Diffusion  processes  have  a  great  deal  of  analytical  structure  and 
cue  therefore  typically  more  mathematically  tractable  than  the  original  process  with  which 
one  starts.  The  approach  underlying  the  application  of  diffusion  approximations  is  there¬ 
fore  comparable  to  that  underlying  normal  approximation  for  sums  of  random  variables. 
In  the  latter  setting,  the  central  limit  theorem  permits  one  to  replace  the  analytically 
intractable  sum  of  random  variables  by  an  appropriately  chosen  normal  random  variable. 

In  this  chapter,  we  shall  describe  some  of  the  basic  theory  of  weak  convergence  that 
underlies  the  method  of  diffusion  approximation.  We  shall  then  survey  various  applications 
of  this  methodology  to  the  approximation  of  complex  queueing  systems.  The  chapter  is 
organized  as  follows: 

1.  Introduction 

2.  Weak  Convergence  of  Stochastic  Processes 

3.  Verification  Criteria  for  Weak  Convergence  of  bt  '■■rh'-tic  Processes 

4.  Donsker’s  Theorem 

5.  Weak  Convergence  Theorems  for  the  Single- Server  Queue 

6.  Background  on  Diffusion  Processes 

7.  Diffusion  Approximations  for  an  Open  Network  of  Queues  in  Heavy  Traffic 

8.  Diffusion  Approximations  for  a  Closed  Network  of  Queues  in  Heavy  Traffic 

9.  Approximations  for  Queues  with  Many  Servers 

10.  Conditional  Weak  Convergence  Theorems 


Because  we  are  interested  in  developing  approximations  for  the  distribution  of  a  process 
(considered  as  a  random  function  of  time),  it  is  necessary  for  us  to  describe  the  basic 
elements  of  the  theory  of  weak  convergence  in  a  function  space.  Sections  2  and  3  are 
therefore  devoted  to  this  topic.  Section  4  discusses  the  most  basic  and  easily  understood 
of  all  diffusion  approximations,  namely  the  general  principle  that  sums  of  random  variables 
(when  viewed  as  stochastic  processes)  can  be  approximated  by  Brownian  motion;  this  result 
is  known,  in  the  literature,  as  Donsker's  theorem.  By  using  the  close  correspondence 
between  random  walk  and  the  single-server  queue,  Section  5  develops  the  basic  theory  of 
weak  convergence  for  the  GI/G/  1/oc  queue.  This  forms  valuable  background  for  the  more 
complex  diffusion  approximations  that  appear  in  the  network  setting  of  Sections  7  and  3. 
Section  6  gives  a  brief  overview  of  some  of  the  basic  analytical  theory  of  diffusion  processes. 
In  particular,  we  show  that  a  large  number  of  interesting  performance  measures  can  be 
calculated  as  solutions  to  certain  associated  partial  differential  equations. 

The  next  three  sections  describe  various  applicatipns  of  the  theory  of  weak  convergence 
to  a  network  of  queues.  In  Sections  7  and  8,  diffusion  approximations  for  both  open  and 
closed  networks  of  queues  in  heavy  traffic  are  given.  The  complex  behavior  of  the  queueing 
model  is  replaced  by  a  more  tractable  “Brownian  network.”  By  Brownian  network,  we  refer 
to  a  diffusion  process  which  is  obtained  by  subjecting  a  Brownian  motion  to  reflection  at 
the  boundaries  of  an  appropriately  defined  region  which  occurs  as  the  “limiting  state 
space"  of  the  queueing  network.  Section  9  discusses  further  weak  convergence  theorems 
that  describe  the  behavior  of  queueing  networks  in  which  the  number  of  servers  is  large. 
The  limit  processes  that  arise  in  this  setting  are  quite  different  from  the  Brownian  networks 
of  Sections  7  and  8. 

Section  10  concludes  the  chapter  with  a  brief  description  of  the  notion  of  conditional 
weak  convergence  theorems.  Several  conditional  limit  theorems  for  the  single-server  queue 
are  described. 

Although  this  chapter  concentrates  on  describing  the  applications  of  diffusion  approx¬ 
imations  to  queueing  networks,  there  are  additional  operations  research  applications  that 
have  been  enriched  by  the  theory.  We  list  several  such  areas  here,  to  give  the  reader  a  flavor 
of  the  broad  impact  that  these  methods  have  had  on  operations  research.  One  important 
example  is  the  application  of  geometric  Brownian  motion  to  the  optimization  of  financial 
gain  from  trading  securities.  The  resulting  Black-Scholes  option  pricing  formula  has  hac  a 


major  impact  on  the  theory  of  finance;  see  DUFFIE  and  PROTTER  (1988)  for  a  descrip¬ 
tion  of  the  basic  limit  theorem.  Another  applications  area  that  has  been  impacted  by  the 
theory  of  diffusion  approximation  is  storage  theory.  YAM  AD  A  (1984)  obtains  diffusion 
approximations  for  a  class  of  storage  systems  characterized  by  a  nonlinear  release  rule. 
The  theory  of  diffusion  approximation  has  also  benefitted  discrete-event  simulation.  For 
example,  one  important  output  analysis  technique,  known  as  standardized  time  series,  has 
been  developed  on  the  basis  of  the  approximation  associated  with  Donsker's  theorem.  See 
SCHRUBEN  (1983)  and  GLYNN  and  IGLEHART  (1989)  for  details.  The  above  is  but  a 
partial  list  of  the  various  operations  rsearch  applications  to  which  the  theory  of  diffusion 
approximation  has  contributed. 

As  stated  above,  this  chapter  focuses  on  providing  an  overview  of  some  of  the  key 
features  of  diffusion  approximation  theory,  as  it  applies  to  queues.  It  is  not  intended  to 
serve  as  a  historical  perspective  on  the  development  of  the  field.  The  author  apologizes, 
in  advance,  to  the  many  contributors  to  the  area  that  have  not  been  adequately  cited. 

2.  Weak  Convergence  of  Stochastic  Processes 

Suppose  that  Y  =  {y(<)  :  t  >  0}  is  a  complex,  analytically  intractable  stochastic  process. 
The  idea  underlying  a  diffusion  approximation  is  to  find  a  diffusion  process  X  such  that 
the  distribution  of  Y  may  be  approximated  by  that  of  X.  Specifically,  we  write  this  as 

(2.1)  yZx 

(  »s  denoted  “approximately  equal  in  distribution  to”).  To  make  more  precise  sense  of 

(2.1) ,  the  standard  approach  is  to  phrase  the  approximation  in  terms  of  a  limit  theorem. 
In  other  words,  we  suppose  that  there  exists  a  sequence  {Xn  :  n  >  0}  of  stochastic  processes 
Xn  =  {X(0  :  t  >  0}  such  that  Y  may  be  identified  with  Xn,  where  n  is  large.  Then,  the 
precise  meaning  of  (2.1)  is  that  the  limit  theorem 

(2.2)  X,  =>  X 

holds,  where  =»  denotes  “convergence  in  distribution”.  The  remainder  of  this  section  is  de¬ 
voted  to  describing  in  more  detail  the  notion  of  convergence  (2.2);  this  type  of  convergence 
is  frequently  termed  weak  convergence. 

We  start  by  reviewing  the  notion  of  weak  convergence  of  random  variables  on  the  rea. 
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line.  A  sequence  {A„  :  n  >  0}  of  r.v.'s  is  said  to  converge  weakly  to  A  if 
(2  3)  P{A„  <  r)  -  P{X  <  *} 

as  n  —  oo  at  all  continuity  points  *  of  the  limit  distribution  function  P{X  <  •}.  This 
convergence  notion  can  be  reformulated  in  many  different  ways.  The  reformulation  which 
we  shall  find  convenient  here  is  given  by  the  following  proposition. 

Proposition  1.  A  sequence  of  r.v.’s  {Xn  :  n  >  0}  converges  weakly  to  X  if  and  only  if  there 
exists  a  probability  space  (ft.F.P)  supporting  a  family  of  r.v.’s  {A'.a;  n  >  0}  such  that: 

i)  An  =  A^  (i.e.  Xn  and  X'n  have  the  same  distribution  for  each  n  >  0). 

ii)  A  =  A'  (i.e.  A'  and  A"  have  the  same  distribution). 

iii)  A'  —  A'  a  s.  as  n  —  x  on  (fi./,  P). 

To  gain  some  insight  into  Proposition  1,  observe  that  if  one  is  given  weak  convergence 
of  Xn  to  X.  then  one  may  construct  the  r.v.'s  A;,A'  in  the  following  way.  Let  (Q.F.P\ 
support  uniform  distribution  on  [0. 1]  and  set 

^  *  F~l(U) 

X'  s=  F~l(U) 

where  U  is  a  uniform  r.v.,  F~l(z)  =  sup{y  :  F„(y)  <  z),  F-'(z)  =  sup{y  :  F(y)  <  z}.  It  is 
well  known  that  X'n  =  Xn,  X'  1  X.  Furthermore,  it  is  easily  verified  that  if  Xn  =>  A. 
then  F~l(U)  —  F’^U)  a.s.  as  n  —  oc,  yielding  iii).  To  start  from  i)-iii)  and  obtain  weak 
convergence  is  even  more  trivial. 

Proposition  1  indicates  the  approach  that  one  needs  to  take  in  order  to  describe  weak 

convergence  of  stochastic  processes.  Examining  the  proposition,  everything  generalizes 

easily  from  the  r.v.  context  to  the  process  setting  (i.e.  the  setting  in  which  the  random 

elements  A„,A  are  random  functions)  except  iii).  But  iii)  shows  that  in  order  to  describe 

weak  convergence  in  a  function  space  setting,  it  is  first  necessary  to  describe  a  notion  of 

convergence  in  function  space,  so  that  a;  —  A'  makes  sense. 

The  most  natural  way  to  do  this  is  to  define  a  metric  d  on  the  function  space,  so  that 

X‘  —  A'  means  d(X'n,X')  —  0.  We  start  by  describing  the  function  space.  Most  operations 

research  applications  involve  the  study  of  stochastic  processes  having  sample  paths  that 

are  right  continuous  with  left  limits.  This  leads  us  to  consider  here  the  function  space 

q  [o  x)  -  I  i0,  ~~  £  suck  that  w(  )  is  right  continuous  \ 

I  for  even*  t  >  0  and  has  left  limits  at  even’ t  >  0  J 


We  shall  further  require  the  range  E  to  be  Euclidian  space.  This  function  space  is  well 
suited,  for  example,  to  the  study  of  queueing  processes,  in  which  the  process  might  cor¬ 
respond  to  the  queue-length  vector  at  time  t  in  a  queueing  network.  We  turn  now  to  a 
description  of  a  metric  d  which  is  suitable  for  the  space  Dc[0,oo). 

Any  “good”  metric  d  on  the  space  De(O.oo)  should  be  able  to  deal  with  the  following 
two  examples: 

a)  Suppose  that  for  all  T  >  0,  ||xn  -  *||r  -Oasn-oo,  where  ||  •  |fr  is  the  uniform  norm  on 
[0.T]  defined  by 

ll-'llr  =  sup{|*(l)|  :  0  <  f  <  T>. 

Then,  it  should  follow  that  d(x„,x)  —  0  as  n  — « oc. 

b)  Given  that  x„(t)  -  l(t  <  1  -  l/n),z(0  =  I(t  <  1),*„  should  converge  to  x  in  the  metric  d. 
in  the  sense  that  d(xn,x)  -  0  as  n  -  x. 

Example  a)  states  that  if  converges  to  *  “nicely”  (i.e.  in  the  topology  of  uniform 
convergence  on  compact  sets),  this  should  imply  convergence  in  the  metric  d.  Example  b) 
requires  that  the  metric  d  should  be  flexible  enough  to  deal  with  the  difficulties  that  arise 
in  describing  closeness  of  discontinuous  functions.  (Note  that  in  b),  ||z„  -x||t  =  1  for  T  >  2. 1 
The  Skorohod  metric  d  on  De[0,  oo)  has  these  properties  and  is  therefore  well  adapted  to 
the  study  of  £>E[0.oo).  In  order  to  dead  with  problem  b)  above,  one  observes  that  i„  =  ioi, 
where  A„(t)  =  t  +  l/n.  So,  xn  is  merely  x  in  which  the  time  scale  is  changed  from  A(t)  =  t  to 
A„ .  Thus,  the  Skorohod  metric  judges  two  functions  x  and  y  to  be  “close”  if  there  exists 
a  “small”  time  deformation  of  y  such  that  the  time-deformed  y  is  “close”  to  x.  See  pp. 
116-122  of  ETHIER  and  KURTZ  (1986)  for  a  rigorous  description  of  the  Skorohod  metric. 

Assume  that  we  have  a  family  of  processes  {X,  X«  :  n  >  0},  for  which  the  saunple  functions 
live  in  De(0,oo)  (i.e.  X  €  DE(0,oo),Xn  €  De(0,oc)).  Following  Proposition  1,  we  adopt  the 
following  definition  of  weak  convergence  of  X„  to  X. 

Definition  1.  If  X„,X  €  Z?e(0,  <*),  we  say  that  .V„  converges  weakly  to  X  (written  =»  A 
if  there  exists  a  probability  space  (n,^,  P)  supporting  a  family  of  r.v.'s  {X'..Y;  n  >  0}  suer, 
that: 

i)  A'n  I  X'n,  X  1  X' 

hi  <rt.v;.X')  -  0  as  as  n-x,  where  d  is  the  Skorohod  metric  on  De[6.  oc). 


We  remark  that  the  family  [X',X'n  .  n  >  0}  is  known  as  the  Skorohod  representation  of 
{X,Xn  :n>0}. 

The  definition  of  weak  convergence  or  Dc[0,oo)  can  be  recast  in  a  somewhat  different 
form.  Consider  a  function  h  :  Z?£[0,oo)  —  D£[0,oo).  The  function  h  is  said  to  be  continuous 
at  x  if  d{h(xn),  /»(*))  -Oasn-oc  whenever  <f(x„,x)  —  0.  Let  S*  =  {x  €  D£[0.oo) :  h  is  continu¬ 
ous  at  x}.  Observe  that  if  d(X'n,X')  —  0  a.s.  with  P{X'  €  5*}  =  i,  then  cf(/»(XA),/i(.Y'))  —  o  a  s . 
which  in  turn  implies  that  h(Xn )  =>  h(X).  We  therefore  obtain  the  following  proposition, 
known  as  the  continuous  mapping  principle. 

Proposition  2.  Let  S*  be  the  set  of  continuity  points  of  a  map  h  :  £)£[0,oo)  —  D£[0.  sc).  It 
Xn  =>  X  as  n  —  oo  and  P{X  €  5*}  =  1,  then  h(Xn)  =>  h( X)  as  n  —  oo. 

As  we  shall  see  later,  the  continuous  mapping  principle  has  a  wealth  of  applications.  A 
second  variant  of  the  continuous  mapping  principle  is  also  useful.  A  function  h  :  D£(0,  oc)  — 
K*  is  said  to  be  continuous  at  z  €  0£[O,oo)  if  Jj/*(x«)  —  A(z)||  —  0  (||  ||  is  Euclidian  norm  on  ZR-.) 
whenever  d(x„,x)  —  0.  Again,  set  S*  =  (r  €  O£(0,oo)  •.  h  is  continuous  at  x}.  The  following 
result  is  the  analog  of  Proposition  2  for  this  new  class  of  h's. 

Proposition  2.  Let  5*  be  the  set  of  continuity  points  of  the  map  h  :  £>£[0,oo)  —  Rd-  If 
Xn  =>  X  as  n  —  oo  and  P{X  €  S*}  =  1,  then  h(Xn)  =*  h(X)  as  n  —  oo.  (The  weak  convergence 
of  h(Xn)  is  standard  weak  convergence  on  #?*.) 

Because  of  the  importance  of  this  result,  it  is  convenient  to  have  simpler  criteria  for 
verifying  that  P{X  €  5*}  =  l  for  a  given  map  h.  Recall  that  in  the  diffusion  approxima¬ 
tion  setting,  the  limit  process  X  generally  is  a  diffusion  process  having  continuous  paths. 
Therefore,  in  order  to  show  that  P{X  e  S„}  =  l,  it  suffices  to  prove  that  h  is  continuous  at 
each  continuous  function  x  €  0£[O,  oo).  The  following  proposition  is  useful  in  verifying  the 
appropriate  condition. 

Proposition  4.  Suppose  z  €  C£(0,:x),  the  space  of  continuous  functions  x  :  [O.oc)  —  E. 
If  x„  €  0£[O.3c).  then  ddn.x)  -  0  as  ri  -  x  is  equivalent  to  requiring  that  for  each  7  > 
0.  i|xn  -  xjjr  —  0  as  n  —  yz. 

Thus,  if  it  is  known  that  the  limit  process  .Y  €  C£[0.  x).  the  vaiidhy  of  P\.\  €  e-.  ■  = 
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may  be  verified  by  showing  that  !|/»(x„)  -  /»(x)||  —  o  as  n  —  oo  for  each  family  {x,  x„  n  >  0} 
such  that 
i)  X  6  Ce[0,  oc)  . 

ii)  For  each  T  >  0,  ||xB  -  x||t  —  0  as  n  —  oo. 

As  a  consequence  of  this  observation,  it  is  often  unnecessary  to  deal  explicitly  with  the 
Skorohod  metric  d\  instead,  one  can  work  with  the  more  easily  manipulated  topology  of 
uniform  convergence  on  compact  sets. 

Specializing  Proposition  3  to  d  =  1,  Proposition  3  shows  that  if  Xn  =>  X  in  De[ o.x). 
then  h(Xn)  =*  /»(A)  in  JR  whenever  h  :  DE[ 0,oo)  —  R  is  continuous  (i.e.  SK  =  DjO.ac)).  The 
bounded  convergence  theorem  thus  implies  that  if  Xn  =>  X,  then 

(2.4)  Eh(Xn)  -  Eh(X) 

for  all  bounded  continuous  h  :  De[Q, oo)  —  IR.  The  following  theorem  states  that  (2.4)  is 
in  fact  equivalent  to  weak  convergence  in  DJO,:*);  see  ETHIER  and  KURTZ  (1986)  for 
further  discussion. 

Theorem  1.  Suppose  Xn,X  €  De(0,oc)  for  n  >  0.  Then  Xn  =>  X  as  n  —  oo  if  and  only  :f 
Eh(Xn)  —  Eh{X )  as  n  —  oo  for  all  bounded  continuous  functions  h  :  DE[0, oo)  —  JR. 

In  most  references  on  weak  convergence  of  stochastic  processes,  the  reformulation  of 
weak  convergence  suggested  by  Theorem  1  is  in  fact  taken  as  the  definition  of  weak  con¬ 
vergence. 

3.  Verification  Criteria  for  Weak  Convergence  of  Stochastic  Processes 

In  Section  2,  the  notion  of  weak  convergence  of  stochastic  processes  was  made  precise 
Our  goal  in  this  section  is  to  give  some  idea  of  what  is  involved  mathematically  in  proving 
that  a  limit  theorem  Xn  X  holds. 

We  first  observe  that  the  projection  map  jt,,  t„  :  DjO.oc)  —  JR'"*  (recall  that  E  =  Ra 
is  continuous  at  every  x  €  CE[ O.oo),  where 

(„(*)  =  (*(ti) . r(*m)) 

Applying  the  continuous  mapping  principle  expressed  by  Proposition  3.  we  obtain  tr.e 
following  result:  If  .V  €  CfiO.sc)  and  .Yn  =>  X.  then  for  every  collection  t|..  ^  of 


indices,  it  must  be  that 


(3  1)  (AT„(<t), . Vn(ui)  =>  (X(tl),....X(«„)) 

?as  n  —  oo.  Relation  (3.1)  asserts  that  if  the  limit  process  X  has  continuous  paths  (as  is 
the  case  for  a  diffusion  limit  process),  then  the  finite-dimensional  distributions  of  X„  must 
converge  weakly  to  those  of  X. 

One  might  expect  that  (3.1)  is  also  sufficient  to  guarantee  weak  convergence  of  X„  to 
X ,  but  this  is  not  so,  as  the  following  example  illustrates. 

Example  1.  For  U  uniform  on  [0.1],  set 

X«(t)  =  exp(-n(t  -  C)J) 

for  /  >  0.  Note  that  for  0  <  <  <  tm, 

(*n(fi), . s>f.Y(t,) . X(tm)) 

where  X(l)  =  0  for  t  >  0.  Hence,  the  finite-dimensional  distributions  of  Xn  converge  to 
those  of  X ,  where  Xn ,  X  6  Q;[0.oo).  On  the  other  hand,  Xn  does  not  converge  weakly  to 
X.  To  see  this,  consider  h(x)  =  max{|r(OI  :  0  <  f  <  1 } ,  and  observe  that  h  is  continuous  at 
any  jt  €  C£[0,oo).  Since  X  €  C£[0,oc),  the  continuous  mapping  principle  would  assert  that 
h(Xn)  =>  /»(X)  if  Xn  =>  X.  However,  h(X„)  =  1,  whereas  h(X)  =  0,  contradicting  the  weak 
convergence  of  X„  to  X. 

Thus,  the  notion  of  weak  convergence  in  De{ 0,x)  requires  something  more  than  weak 
convergence  of  the  corresponding  finite-dimensional  distributions.  To  argue  that  X«  »  X 
involves  the  following  circle  of  ideas. 

Definition  2.  Given  a  family  {P,  P„  :  n  >  0}  of  probability  measures  on  0  =  Df[0.x).  we 
say  that  Pn  converges  weakly  to  P  (written  P„  ^  P)  if 

/  h(„)Pn(d~)  -  /  />(w)P(<M 
J  n  J  n 

for  ail  bounded  continuous  functions  h  Q  —  R. 

By  Theorem  1.  X„  =>  X  if  and  only  if  Pn  =»■  P.  where  Pn(  )  =  P{X„  €  }  and  P\  )  =  P{.\  € 

It  turns  out  that  weak  convergence  of  the  probability  measures  Pn  to  P  can  be  formulates 


in  terms  of  convergence  in  a  certain  metric  p.  To  be  precise,  let  V  =  {Q  :  Q  is  a  probability 
measure  on  £>e{0,oo)}.  There  exists  a  metric  p  on  V  such  that  Pn  ^  P  as  n  -  oc  is  equivalent 
to  requiring  thatp(Pn,  /,)-0asn-x,  the  metric  p  is  called  the  Prohorov  metric.  Thus, 
the  notion  of  weak  convergence  has  been  recast  in  terms  of  convergence  in  a  certain  metric 
P- 

A  standard  approach  to  proving  that  x„  —  x  when  the  x„’s  and  x  are  elements  of  a 
metric  space  is  to  proceed  via  the  following  two  steps: 

i)  Show  that  the  sequence  {?„  :  n  >  0}  is  relatively  compact,  in  the  sense  that  every 
subsequence  x„.  has  a  further  convergent  subsequence  x„«. 

ii)  Show  that  every  convergent  subsequence  xn.  of  zn  must  converge  to  x. 

The  first  step  shows  that  every  subsequence  has  a  limit  point,  whereas  the  second  step 
proves  that  the  only  possible  limit  point  of  {xn  n  >  0}  is  x.  Thus,  the  second  step  involves 
identifying  the  set  of  all  possible  limit  points.  Returning  to  the  space  V ,  we  wish  to  obtain 
a  condition  for  identifying  the  set  of  limit  points  of  a  sequence  {Pn  :  n  >  0}. 

Theorem  2.  Consider  {P.  P„  n  >  0}  where  P„(  )  =  P{Xn  e  },  /'f  )  =  P{X  e  }  and  A'n  A  e 
De[ Q.x).  If  the  finite-dimensional  distributions  of  Xn  converge  to  those  of  .V  (i.e.  if  (3.1  1 
holds),  then  the  only  possible  limit  point  of  {Pn  n  >  0}  is  P. 

Thus,  Theorem  2  shows  that  convergence  of  the  finite-dimensional  distributions  is 
important  in  identifying  the  set  of  possible  limit  points  of  Pn. 

Returning  to  step  one  of  the  convergence  proof  outlined,  we  need  to  obtain  cntena 
guaranteeing  relative  compactness  of  a  sequence  {Pn  :  n  >  0}  of  probability  measures  in  P 
The  following  theorem,  due  to  Prohorov,  throws  the  question  of  compactness  in  V  back 
mto  determining  compactness  in  De[0,oc). 

Theorem  3.  Consider  {Pn  n  >  0}  where  Pn  €  V-  Then.  {Pn  n  >  0}  is  relatively  compxc: 
in  p  (i.e.  for  every  subsequence  n',  there  exists  a  further  subsequence  n"  and  a  probability 
measure  P"  €  V  such  that  p(/V.P")  -  0  as  n"  -  x)  if  and  only  if  for  every  r  >  0.  there 
exists  a  compact  set  K,  c  D^'O  x)  (i.e.  compact  in  the  metric  d  on  Df[0,x))  such  that 


Because  of  the  obvious  importance  of  the  notion  (3.2),  it  has  received  a  name. 

Definition  3.  A  family  {Pn  €  V  n  >  0}  of  probability  measures  on  D£[0,oc)  is  said  to  be 
tight  if  for  every  t  >  0,  there  exists  a  compact  set  K,  C  Ds[0,  oo)  such  that  mfn>0  Pn(K,)  >  1-c. 

We  can  now  state  the  conventional  “first -principles”  approach  used  to  prove  a  limit 
theorem  of  the  form  Xn  ^  X  when  Xn  e  De[0,oo),A  €  CcfO.oo).  One  first  shows  that  the 
family  {Pn  :  n  >  0}  is  tight,  where  P„( )  =  P{Xn  6  •},  followed  by  proving  that  the  finite¬ 
dimensional  distributions  of  X„  converge  to  those  of  X.  Because  of  the  important  role  of 
tightness  in  proving  limit  theorems  in  De[ 0,oo),  the  characterization  of  the  compact  sets  in 
CftO.Dc)  and  £>£[0, oc)  occupies  a  central  place  in  the  corresponding  limit  theory. 

However,  in  many  operations  research  settings,  one  can  avoid  the  technical  complica¬ 
tions  associated  with  the  above  “first-principles”  argument  by  making  use  of  the  continuous 
mapping  principle.  Suppose  that  one  wishes  to  show  that  Xn  =>  X  and  that  one  can  rep¬ 
resent  Xn,X  as  A'„  =  h(Yn),  X  =  h(Y)  for  some  function  h  such  that  P{Y  e  S*}  =  1.  Then, 
the  weak  convergence  of  Xn  to  X  follows  immediately  if  it  is  known  (either  by  assumption 
or  by  previously  developed  theory)  that  Y„  =>  Y.  We  will  see  this  “continuous  mapping” 
approach  illustrated  in  the  queueing  example  developed  in  Section  5. 

4.  Donsker’s  Theorem 

We  shall  now  describe  the  most  important  functional  limit  theorem  in  the  theory  of 
stochastic  processes.  Let  {Z„  :  n  >  0}  be  an  i.i.d.  sequence  of  lR--valued  r.v.'s  and  set 
S„  =  Z\  +  ■  •  •  +  Zn  with  So  =  0.  Donsker’s  theorem  is  a  limit  theorem  that  describes  the 
behavior  of  the  J^-valued  random  walk  {5n  n  >  0)  over  long  time  intervals;  this  theorem 
can  be  viewed  as  a  “building  block”  for  many  of  the  other  limit  theorems  developed  in 
probability  theory. 

We  first  review  some  of  the  classical  limit  theory  for  the  random  walk  {5n  :  n  >  0}.  We 
start  with  the  strong  law  of  large  numbers,  which  states  that  if  E||Z„||  <  oo,  then 

(4  1)  a.s. 

n 

as  n  —  oc.  w’j>  ■'//=  EZ„.  The  law  of  large  numbers  (4.1)  can  be  refined  by  appealing  to 
the  centra]  .  ..  theorem,  which  asserts  that  if  f||Zn||J  <  oc.  then 

(4  21  n‘n  =>  El/J.V(0./) 
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where  N( 0,  /)  is  an  Revalued  multivariate  normai  r.v.  with  mean  vector  0  and  the  identity 
as  covariance  matrix,  and  El/2  is  the  square  root  of  the  covariance  matrix  E  =  EZ'nZn  - 
EZ'n  EZn ■  (We  assume  here  that  Z„  is  a  row  vector.) 

The  idea  is  now  to  look  for  process- valued  versions  of  (4.1)  and  (4.2).  First,  consider 
X„(t)  =  n“lS(nt],  where  (*]  denotes  the  greatest  integer  less  than  or  equal  to  x.  It  can  be 
shown  that  the  limit  theorem  (4.1)  implies  that 

(4.3)  </(*„.*)-  0  &.s. 

as  n  —  oo,  where  X(t)  =  nt\  (4.3)  is  called  the  functional  strong  law  of  large  numbers 
To  obtain  a  functional  form  of  the  central  limit  theorem,  we  use  the  same  scaling  as  in 
(4.2)  and  consider  the  stochastic  process 

xm = (5=a .  „,) = o  -  X(o). 

Note  that  one  unit  of  time  in  the  process  X„  corresponds  to  n  time  units  of  the  random 
walk,  and  that  one  spatial  unit  of  X„  is  equivalent  to  n1/J  spatial  units  of  the  random  walk. 
To  identify  the  limit  behavior  of  X„ ,  note  that  (4.2)  proves  that 

(4  4)  *„(<)=>  E1/J(V(0,t-/) 


for  each  t  >  0.  Furthermore,  the  stationary  independent  increments  of  Xn  allows  one  to 
extend  (4.4)  to  the  finite-dimensional  distributions  of  X*:  For  0  <  tj  <  tj  <  <  t„. 

(*„(<,) . A-n(tn)) 

=>(A-(ti) . X(tm)) 

as  n  —  oo,  where  (X(ti),...,X(tm))  is  a  Gaussian  (i.e.  multivariate  normal)  random  vector 
with  mean  and  covariance  described  by 


(4.5) 


EX(t,)  =  0.  1  <  i  <  m 
EX'(t,)X(t,)  =  Emin(t,.t;). 


The  only  process  supported  on  De[0,3c)  with  finite-dimensional  distributions  as  described 
by  (4.5)  is  the  rirownian  motion  process.  Thus,  if  X  €  Dg[ 0,<x>),  it  must  be  that  X  = 
where  B(  )  is  a  standard  Brownian  motion  process  on  R4  having  the  following  properties 

1 1  B(  )  €  Ce(0,3c) 

ii  i  Bl  )  has  stationary  independent  increments 
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iii)  fl(<)  =  tf(0,r/). 

The  following  result,  known  both  as  Donsker’s  theorem  and  the  functional  central  limit 
theorem  (FCLT),  is  a  precise  statement  of  the  limit  behavior  of  the  process  {Xn  n  >  0}. 

Theorem  4.  If  E\\Zn\\7  <  oo,  then  Xn  ^  El/Jfl  as  n  —  oo  in  D£[0,oo). 

We  remark  that  Theorem  4  is  the  prototypical  diffusion  approximation,  in  that  E1/J0 
is  a  diffusion  process.  (In  fact,  it  is  fair  to  say  that  B  is  the  most  fundamental  diffusion 
process.)  Theorem  4  illustrates  several  important  features  that  are  common  to  diffusion 
approximations.  First,  observe  that  the  approximation  suggested  by  the  limit  theorem 
takes  the  form 

Thus,  the  limit  process  that  is  used  to  approximate  Stn,]  depends  on  the  random  walk  only- 
through  the  mean  vector  n  =  EZn  and  the  covariance  matrix  E  of  Z„. 

Secondly,  a  great  deal  of  information  about  Xn  may  be  inferred  from  Theorem  4  by 
taking  advantage  of  the  continuous  mapping  principle.  We  illustrate  the  power  of  this  idea 
by  offering  the  following  examples. 

Example  2.  Assume  the  random  walk  is  real- valued  (i.e.  d  =  1)  with  n  =  0  and  consider 
the  following  functional  of  {5„  :  n  >  0}  :  m«{S*  :  0  <  ir  <  n}.  We  observe  that  this  functional 
may  be  expressed  in  terms  of  an  appropriately  chosen  mapping  h  defined  on  X„  namely 

max {5k  :  0  <  t  <  n}  =  nl/2h(Xn) 

where  h(x)  =  max{x(t)  .  0  <  t  <  1}.  By  applying  Proposition  4,  it  is  easily  verified  that  h 
is  continuous  at  any  x  e  Ce[0,oo).  Thus,  since  Brownian  motion  has  continuous  paths,  we 
conclude  that  P{X  €  5a}  =  1,  where  X  =  oB  and  <rJ  =  varZ„.  Hence,  by  Proposition  3  and 
Theorem  4,  h(Xn)  ^  h(vB)  and  we  obtain  the  approximatic 

max {5*  :  0  <  k  <  n}  as  <rnl/J  max{5(t)  :  0  <  t  <  1}. 

But  by  using  the  “reflection  principle”  (see  KARLIN  and  TAYLOR  (1975)).  one  can  ex¬ 
plicitly  calculate  the  distribution  of  h(B ).  namely 

P{ max {£?;<)  0  <  <  <  1}  >  x)  =  /  exp(-t:/2)dt. 


(46) 


Example  3.  For  the  same  random  walk  as  in  Example  2,  consider  #{*  :  0  <  *  <  n,Sk  >  o). 
Observe  that 

#{±  :  0  <  i  <  n,S*  >  0}  =  n/ifX,) 

where  h{x)  =  /0‘  l(x(t)  >  0 )dt.  Again,  one  can  easily  verify  that  P{oB  €  S»)  =  1,  by  which  we 
may  conclude  that  A(A„)  h(oB)  -  h(B).  Again,  it  is  possible,  using  properties  of  Brownian 
motion,  to  explicitly  calculate  the  distribution  of  h(B): 

P{h(B)  <  x}  = -ucsinx1^,  0  <  x  <  1. 

This  limit  theorem  comprises  the  well  known  “arcsin  law”  for  random  walk. 

Example  4.  For  the  random  walk  {S„  :  n  >  0}  of  Example  2,  consider  the  hitting  time 
T(a)  =  min{fc  >  0  :  5*  >  a}, a  >  0.  To  analyze  the  distribution  of  the  hitting  time,  observe 
that  {7(a)  >  n}  =  {m&x{S*  0  <  k  <n)  <  a).  Hence,  { T(an l/»)  >  [in]}  =  {m&x{S*  :  0  <  k  <  [nx]}  < 
an1/J}  and  one  can  appeal  to  Example  2  to  show  that 

P{7(an1/J)  >  [xn]}  —  P{max{B(t)  :  0  <  t  <  x}  < 

<7 

which  can  be  calculated  from  (4.6). 

Example  5.  This  example  will  serve  to  illustrate  the  point  made  at  the  end  of  Section 
3,  namely  that  many  functional  limit  theorems  can  be  derived  by  judicious  use  of  the 
continuous  mapping  principle,  thereby  avoiding  a  “first  principles”  argument  For  the 
random  walk  of  Example  2,  set 

j  \0,  <>1. 

*n(0  =  n-1/S5n(0- 

Note  that  Xn  =  h(Xn),  where  h  :  De[0,oo)  —  D£[0,oo)  is  given  by  h(x)(t)  =  x(t)-fx(l)  for  0  <  t  <  l 
and  h(x)(t)  =  0  for  t  >  1.  It  is  easily  verified,  using  Proposition  4,  that  h  is  continuous  at  any 
x  €  CcfO.oc),  so  that  Proposition  2  allows  us  to  conclude  that  h(Xn)  =>  h(<rB).  We  remark 
that  the  process  B(t)  -  tfl(l)(0  <  t  <  1)  is  the  so-called  Brownian  bridge  process  (also 
known  as  "tied-down"  Brownian  motion). 

An  additional  remark,  related  to  Theorem  4.  is  in  order.  For  an  appropriately  con¬ 
tinuous  h,  the  continuous  mapping  principle  proves  that  /»(Xn)  =»■  h(cB)  as  n  —  x.  One 
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way  to  calculate  the  distribution  of  h(<rB)  is  to  use  this  limit  theorem  to  observe  that  the 
limit  random  element  h(oB)  is  invariant  with  respect  to  the  particular  choice  of  the  random 
walk  {S«  :  n  >  0}  (modulo  <rJ  =  va tZ„).  Thus,  one  may  calculate  the  distribution  of  h(oB) 
.by  choosing  a  particularly  simple  random  walk  (symmetric  nearest  neighbor  random  walk 
on  the  integers  is  a  typical  choice),  and  taking  the  limit  of  k(Xn)  for  the  chosen  random 
walk.  This  approach  to  calculating  the  distribution  of  h(aB)  explains  why  Theorem  4  is 
often  called  the  invariance  principle. 

A  number  of  important  extensions  to  the  basic  FCLT  described  by  Theorem  4  have 
been  made  over  the  years.  The  flavor  of  most  of  these  extensions  has  been  to  prove  limit 
theorems  for  the  partial  sum  process  in  which  the  summand  sequence  {?„•*»>  0}  is  no 
longer  i.i.d.,  but  instead  allows  for  some  kind  of  dependence  and  (possibly)  some  mild 
form  of  non-stationarity.  Since  partial  sum  processes  often  serve  as  “inputs"  to  various 
stochastic  processes  arising  in  operations  research  (e.g.  if  the  Z,’ s  correspond  to  inter¬ 
arrival  times  in  a  queue,  the  S„’s  are  the  arrival  times),  we  will  now  describe  some  of  the 
extensions  that  are  available.  We  specialize  to  the  setting  in  which  the  Z,’s  are  real- valued, 
to  avoid  complications  in  stating  results,  but  note  that  similar  limit  theorems  hold  in  the 
Rd- valued  vector  context. 

The  generic  form  of  the  FCLT  involves  a  statement  that,  for  a  given  (real- valued ) 
sequence  {Z„  :  n  >  0},  there  exist  finite  constants  n  and  a  such  that 

(4-  Xn=><rB 

in  £>s[0,x),  where 

The  extensions  of  the  FCLT  that  are  available  include: 

a)  {Zn  :  n  >  0}  is  a  stationary  mixing  process.  Here,  {Zn  n>  0}  is  a  strictly  stationary 
sequence  which  satisfies  a  mixing  condition.  Roughly  speaking,  a  mixing  hypothesis 
states  that  events  which  occur  at  widely  separated  time  points  are  asymptotically 
independent  of  one  another.  This  is  generally  stated  mathematically  as  a  condition  of 
the  form 

P{Z0m  €  A.zz::*r  efi)  -  P(Z0m  €  A)P{Zr0  €  B } 

as  n  —  x,  where  Z\  -  (Z, . Z;)  for  i  <  j.  Assuming  that  EZ\  <  x  and  tr.a: 

nLo'coMZo.Z*)!  <  x.  the  relevant  FCLT's  yield  limit  theorems  of  the  form  <4  >  . 
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where  n,  o  are  given  by 


H  =  EXn 


<r7  =  var  Z0  +  2  ^2  cov(^o.  Z*). 

-  •  k*l 

For  a  more  precise  formulation  of  these  results,  see  ETHIER  and  KURTZ  (1986),  p. 

350-353. 

b)  {Sn  :  n  >  0}  is  a  martingale  sequence  with  stationary  martingale  differences.  (See  the 
handbook  chapter  on  martingales.)  If  {Sn  .  n  >  0}  is  a  martingale  sequence  for  which 
the  differences  {£>„  :  n  >  1}  (D„  =  S„  -  Sn-i)  are  strictly  stationary,  then  (4.7)  holds  if 
ED 7  <  3o  and  n, a  are  given  by  /j  =  0,  <r2  =  ED*. 

c)  (Zn  :  n  >  0}  is  a  regenerative  sequence  with  regeneration  times  0  <T0<Tl<-.  If 

E(TX  -  T0)J  <  oo 

r(T£|z’*1)  <  3°' 

then  (4.7)  is  valid  with  ji  =  E  Zn/E(TX  -  r0)  and  <rl  =  E  (El&(*»  -  (*))  ’  /W  -  T0 ); 
see  GLYNN  and  WHITT  (1987). 

d)  {Zn  :  n  >  0}  is  a  real-valueu  functional  of  a  time-homogeneous  positive  recurrent 
Markov  chain.  Thus,  Z„  =  /(Wn)  where  {Wn  :  n  >  0}  is  a  time-homogeneous  Markov 
chain  satisfying  a  suitable  recurrence  condition;  let  »(•)  be  the  associated  stationary 
distribution  of  {Wn  :  n  >  o).  We  remark  that  there  is  no  need  here  for  Wn  to  be  discrete¬ 
valued.  Under  certain  regularity  hypotheses  (see  NUMMELIN  (1984)  for  a  precise 
description  of  the  results),  (4.7)  holds  with 

n  =  Etf(Wn) 

00 

<rJ  =  E,(f(W0)  -  n)7  +  2  Y.  EAf(W0)  -  -  ?) 

ksl 

(£■,(  )  is  the  expectation  on  the  path  space  of  (Wn  :  n  >  0)  under  which  W0  has  distri¬ 
bution  if. ) 

The  above  FCLT’s  are  an  indication  of  the  basic  robustness  of  the  Brownian  motion  ap¬ 
proximation  for  the  partial  sum  process.  They  indicate  that,  regardless  of  the  fine  structure 
of  the  summands,  that  partial  sum  processes  over  long  time  intervals  behave  (when  suitably 
normalized)  like  Brownian  motions. 


We  conclude  this  section  with  the  discussion  of  a  very  important  and  powerful  theorem 
which  states,  in  a  very  strong  sense,  that  random  walk  is  well  approximated  by  Brownian 
motion.  The  type  of  theorem  that  we  shall  discuss  here  is  called  a  strong  approximation 
theorem,  since  it  is  phrased  not  in  a  distributional  sense  (as  in  (4.7))  but  in  an  “almost 
sure”  sense. 

The  following  result,  which  is  due  to  K0ML6S,  MAJOR,  and  TUSNADY  (1975).  is  a 
particularly  sharp  form  of  the  strong  approximation  theorem  for  random  walk. 

Theorem  5.  Let  {Z„  :  n  >  1}  be  a  sequence  of  i.i.d.  real-valued  r.v.’s  satisfying  Eexp{aZn )  < 

x  for  a  in  an  open  neighborhood  of  zero.  Set  So  =  0  and  SB  -  Z h - vZn  for  n  >  1.  Then  there 

exists  a  probability  space  (Q.X.P)  supporting  a  standard  Brownian  motion  {B(t)  :  t  >  0} 
and  a  sequence  {S^  :  n  >  0}  such  that: 

i)  {s;  :  n  >  0}  1  {S„  :  n  >  0}  (i.e.  the  sequence  {SB}  shares  the  same  distribution  as  {S„})- 

ii)  For  every  z  and  n. 

P  (  max  | Si  -  kft  -  <rB(i)|  >  Clogn  +  z\  <  Ke~Xx 

J 

where  /j  =  EZn ,  <rJ  =  varZn,  and  K,C,  and  X  are  positive  constants  depending  only  on 
the  distribution  of  Z„. 


It  is  easily  verified  (use  the  Borel-Cantelli  lemma)  that  ii)  implies  that 


(4.8) 


SJ,  =  nn  +  <rB(n)  +  0(log  n)a.s. 


(The  sequence  0(logn)  represents  a  sequence  {&, :  n  >  0}  of  r.v.’s  such  that  |rtn|  <  Alogn  +  B 
where  A,  B  are  finite- valued  r.v.’s.)  This  result  is  sharp,  in  the  sense  that  if  S^  =  n^+<rB(n)- 
o(logn)  a.t.,  then  Zn  has  a  JV(^,c3)  distribution  so  that  the  o(togn)  term  can  be  taken  to  be 
zero. 

It  is  easily  verified  that  (4.8)  implies  Theorem  4.  Set 


*;«)  =  r. 


-  n!/J 


and  note  that  by  i).  X'n  1  Xn.  By  (4.8)  and  basic  properties  of  Brownian  motion. 


(4.9)  TO-)  -  n-1/3<rS(n  )jjr  =  0(logn/n1/s)  a.s. 

for  any  T  >  0.  But  n'inB(n  )  1  B(  ).  Hence.  Theorem  4  follows  from  (4.9)  by  lettine  n  -  x 
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One  important  application  of  Theorem  5  is  to  (easily)  obtain  rates  of  convergence 
for  various  limit  theorems  related  to  Donsker’s  theorem.  Consider  Example  2,  in  which 
it  is  shown  that'A(X„)  =>  h(erB)  where  h(x)  =  max{x(t)  :  0  <  t  <  1}.  By  i),  h(Xn)  1  h{X'n). 
Straightforward  analysis,  using  ii),  then  proves  that 

P{HK)  <  *}  =  PM°B)  <  *)  +  0  . 

As  in  the  case  of  Theorem  4,  there  exist  extensions  of  strong  approximation  results  to 
dependent  sequences;  a  good  reference  for  such  theorems  is  PHILIPP  and  STOUT  (1975). 

5.  Weak  Convergence  Theorems  for  the  Single-Server  Queue 

In  this  section,  we  indicate  how  the  weak  convergence  theory  of  Sections  2  through 
4  can  be  applied  to  obtain  limit  theorems  for  the  single-server  queue.  To  be  precise,  we 
consider  a  single  GI/G/ l/oo  queueing  system  in  which  customer  0  arrives  at  time  T0  =  0. 
finds  a  free  server,  and  experiences  a  service  time  V0.  .The  n’th  customer  arrives  at  time  Tn 
and  experiences  a  service  time  V„.  Setting  Un  ~Tn-  r„_i  (n  >  1),  we  assume  that  each  of 
the  sequences  {Un  .  n  >  1}  and  {14,  n  >  0}  are  i.i.d.  and  independent  of  one  another.  The 
three  processes  that  we  shall  consider  in  this  section  are: 

Wn  =  waiting  time  (i.e.  excluding  service)  of  the  nth  customer 
Q(t)  =  number  of  customers  in  the  system  (i.e.  including  the  server)  at  time  t 
D(t)  =  the  cumulative  number  of  customers  departing  the  system  in  [0,t]. 

Finally,  let  EUn  =  X~l,EVn  s  p~l,  where  0  <  A,*i  <  oo. 

We  shall  first  discuss  the  relevant  theory  when  the  traffic  intensity  p  ~  \/p  <  1.  In  this 
“light  traffic”  setting,  the  queue  is  stable  in  the  sense  that  { Wn  :  n  >  0}  and  {Q(t)  :  t  >  0} 
are  stochastically  bounded.  In  fact,  it  can  be  shown  that  Wn  s»  W  as  n  —  oo,  where  the 
characteristic  function  of  W  is  given  by 

Fexp(itlV)  =  exp  n” 1  [£7(exp(*'<5^ )  -  1)]|  , 

and  Sn  =  £";0‘r,  -  J3"_j  U,  (n  >  1)  with  50  =  0  {S*  =  max(5n.0)).  Furthermore,  if  we 
additionally  assume  that  Un  has  a  non-lattice  distribution,  then  there  exists  a  proper 
t  v.  Q  such  that  Q{t)  (J  as  !  -  x.  Given  the  well-behaved  nature  of  Wn  (and  Q{t)).  it 
is  easy  to  show  that  there  do  not  exist  sequences  {an}  and  {6n}  for  whicn  the  ranaom 
functions  {(W'!n,j  (and  -  a„t)/M)  converge  weakly  to  a  non-degenerate 

i; 


process  in  0|t(O,oo).  This  suggests  that  we  should  change  our  point  of  view,  to  consider 
the  cumulative  processes 

n 

T.  and 
j=i 

The  analysis  of  these  cumulative  processes  depends  on  the  fact  that  both  {Wn  :  n  >  0} 
and  (Q(t)  :  t  >  0}  are  regenerative;  this  allows  us  to  apply  the  FCLT  for  regenerative 
processes  (discussed  in  Section  4)  to  obtain  limit  behavior  for  the  cumulative  processes. 
To  precisely  state  the  relevant  theorems,  let  ij  =  inf{n  >  1  :  Wn  =  0}  and  let  T  =  inf{t  > 
«0  :  Q(t-)  =  0,  Q(t)  a  1};  n  and  T  are  regeneration  times  for  {Wn  :  n  >  0}  and  (Q(t)  :  t  >  0} 
respectively. 


i: 


Q(,)d,. 


Theorem  6.  Suppose  p  <  1.  If  El l  <  x.EVf  <  oc,  £V  <  x,ET2  <  oc.f  <  :x' 

and  E  (/0r  Q(s)ds'j  <  x ,  then 


n1'2 


Q(t)di  -  tEQ'j 


=>  ffiB(t) 

=> 


in  Dk[0,oo)  where  <r\  =  E  -  f’W))  /Erj  and  where  a\  =  E  (/0T(<?(«)  -  EQ)<d*)  /ET- 

Furthermore,  EW  =  E  YilZa  Wk/Etj  and  EQ  =  E  /0T  Q(t)ds/ET. 


Recall  that  when  p  <  1,  the  translation  constants  EW  and  EQ  appearing  in  Theorem 
6  are  related  via  Little’s  formula:  EQ  =  XEW.  A  similar  “Little-type”  result  holds  for  the 
variance  constants  <r?  and  <r|;  for  details,  see  GLYNN  and  WHITT  (1986). 

Turning  now  to  the  departure  process,  we  observe  that  D{t)  =  N(t)  -  Q(t)  where  -V(<)  = 
number  of  arrivals  by  time  t.  Note  that  N(t)  is  essentially  the  renewal  process  corresponding 
to  the  sequence  {Un  ■  n  >  l}.  To  handle  .V(t),  we  use  a  basic  idea  from  the  theory  of 
weak  convergence  of  stochastic  processes,  namely  the  method  of  “random  time  change”. 
Evidently,  N(i),  when  viewed  on  the  time  scale  of  the  r„’s,  is  deterministic,  by  which  we 
mean  that  N(Tn)  =  n  +  i.  Similarly,  the  sequence  {Tn  .  n  >  0},  when  viewed  through  .Vm  s 
time  scale,  is  basically  non-random,  in  that  TN(t)  as  *.  Thus,  {r*}  and  {N(t)}  are  "inverse 
processes  to  one  another.  This  important  idea  is  central  to  the  study  of  ff{  ). 

To  exploit  these  "time  change"  ideas  in  the  weak  convergence  setting,  observe  that  a 
"time  change"  corresponds  to  composition  of  processes.  Thinking  of  as  a  change  in  time 
scale  (i.e.  from  t  to  <t>n(0).  need  to  consider  when  .Y„  o$n  converges  weakly  to  .V  o<t> 
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In  the  following  propositions,  we  endow  the  product  space  with  the  product  metric 
induced  from  the  component  spaces  (i.e.,  a  sequence  converges  in  the  product  space  if  and 
only  if  the  components  converge  in  their  respective  spaces). 

Proposition  5.  Suppose  A„  is  E- valued  and  :  [0,oo)  —  (0,oo)  is  non-decreasing.  If 
(A„.<&„)  =>  (A,$)  in  0£[O.oo)  x  D*[0,oo)  where  (A-,*)  €  Cg[0,oo)  x  C *,  then  Xn  <>♦«=>  A  o*  in 
D£[0,oo). 

See  BILLINGSL£Y  (1968),  Section  17,  for  additional  details.  To  establish  the  joint 
convergence  in  Proposition  5,  the  following  tool  is  often  used. 


Proposition  6.  Suppose  that  X'n  e  D£.[0.3c)  and  that  there  exists  (deterministic)  x  € 
Z?£'[0. »)  such  that  d(A;,x')  0  in  R.  (This  is  equivalent  to  A*  =>  z‘  in  £>£r-(0, oo). )  If  An  =>  A 

in  0£[O,oo),  then  (Xn,X'n)  =>(A ,*')  in  £>£[0,oo)  x  0£<(O,oo). 


To  apply  these  results  to  A’(  ),  one  first  generalizes  the  strong  law  for  renewal  processes 
to  a  functional  strong  law: 


,N(nt) 


-  At|(r  0  aj. 


as  n  —  for  each  T  >  0.  Since  At  is  deterministic,  Proposition  6  and  Donsker’s  theorem 
for  the  inter-arrival  times  yields  the  weak  convergence  relation 

("l/’  -  A"1,)  .  =>  («r4B(*).A#) 

where  a\  =  var  U  the  random  time  change  result  provides 

"l/2  (n<W(n<)  “  ** 


Since  Ts{n»)  «*,  one  obtains  the  following  FCLT  for  N(-): 

(5  1)  nl^J  ^  <r4A3^7fl(j) 

(we've  used  the  fact  that  B( a  )  =  qU7B( •)).  To  deal  with  the  approximation  rV(„,,  *  ns.  one 
uses  the  following  “converging-together"  result. 


Proposition  7.  If  A„  =>  A  in  D£{0.3c)  and  d(Xn,  X'n)  ^  0  in  R,  then  A;  ^  A  in  D£[0.x  i. 

i  For  Propositions  6  and  7.  we've  implicitly  assumed  that  An  and  X'n  are  defined  or. 
the  same  probability  space.)  Propositions  5  through  7  follow  directly  from  the  Skoroho' 
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representation  and  the  fact  that  the  Skorohod  topology  reduces  to  the  topology  of  uniform 
convergence  when  the  limit  elements  are  continuous.  The  ideas  exploited  above,  namely 
those  of  “continuous  mapping”,  “random  time  change”,  and  “convergence  together”,  are 
ithe  three  main  tools  used  in  proving  most  of  the  diffusion  approximations  that  arise  in 
operations  research.  The  limit  theorem  (5.1)  is  a  particularly  simple  application  of  these 
ideas,  but  is  nevertheless  somewhat  typical  of  how  diffusion  approximation  limit  theorems 
are  proved  in  operations  research. 

We  return  now  to  the  departure  process  D{t).  Since  the  queue-length  process  Q(  )  is 
stochastically  bounded,  the  next  result  follows  from  (5.1)  and  the  “converging  together" 
proposition. 

Theorem  7.  Suppose  p  <  l  and  EU 3  <  x.  Then, 

in  £>*[0,oo). 

This  concludes  our  discussion  of  the  stable  case  in  which  p  <  1;  we  turn  now  to  analysis 
of  the  unstable  GI/G/ l/oo  queue,  in  which  the  traffic  intensity  p  >  1.  The  crucial  point 
here  is  that  when  a  queue  is  oversaturated,  there  exists  some  (random)  time  3  after  which 
the  server  is  never  idle.  One  now  observes  that  if  S(t)  is  the  renewal  counting  process 
corresponding  to  the  service  times  Vo,  Vi , ... ,  then  S(t)  can  be  interpreted  as  the  total  number 
of  customers  that  would  exit  by  t  if  the  server  were  in  constant  operation.  Thus,  D{t)  as  S(t) 
and  <?(t)  as  N(t)  -  S(t),  from  which  the  next  result  follows  (use  (5.1)  and  the  independence 
of  ,V(  ),5(  )). 

Theorem  8.  Suppose  p  >  l  and  EU\  <  oo,  £%’  <  oo.  Then 

B(|) 

(2M  -  (A  -  „w)  ^ 
in  £>*[0.  dc ),  where  <t\  =  var  Vn. 

An  immediate  consequence  of  Theorem  8  is  the  following  “pplication  of  the  continuous 
mapping  principle:  if  >  0.  then 

P{Q(n)  <  n(X  —  p)  +  n;/ V^A3  +  cr|^i3)l/2z) 


as  n  — .  oo.  To  handle  the  waiting  times,  we  use  the  fact  that  the  waiting  time  sequence 
{Wn  :  n  >  0}  satisfies  the  recursion  Wn+X  =  [Wn  +  Xn+I]+  (with  W0  =  o),  where  Xn  =  t;.!  -  i\. 
Since  the  system  is  oversaturated,  Wn  —  x  a  s.  and  is  hence  positive  with  high  probability 
for  large  n.  Thus,  Wn^  as  Wn  +  Xn.fi  (i.e.  the  positive  part  operator  can  be  dropped).  This 
implies  that  Wn+\  «  S„+i,when  S«+i  =  Xi  +  — t-  -Vn+i.  A  limit  theorem  for  Wn  can  then  be 
obtained  from  the  FCLT  for  S„. 

Theorem  9.  Suppose  p  >  1  and  EUl  <  oo.  EV*  <  oo.  Then 

n1'1  =►  [a\  +  <r|)l'lB(t) 

in  D«[0.  oc). 


We  now  describe  the  approximation  theorems  that  are  available  in  the  setting  of  "heavy 
traffic”,  where  p  =  1.  Returning  to  the  waiting  time  sequence  {Wn  :  n  >  0},  the  recursion  for 
the  tVn’s  can  be  solved  to  yield  Wn  =  S„  -  min{5»  0  <  k  <  n},  from  which  it  follows  that 


'  ~  nun  ;  0  <  *  <  *  j 


=  /(XnM<) 


when  *„(<)  =  5[„ij/n1/2  and  /  0*[O,  oc)  —  Dn[ 0.  oo)  is  given  by 
(5.2)  /(x)(t)  =  *(<)  -  min{x(x)  :  0  <  »  <  <}. 

By  Donsker's  theorem,  S„  =>  (c\  +  (r|)1/JB.  On  the  other  hand,  it  is  easily  verified  that 
/  is  continuous  in  the  Skorohod  topology  at  any  continuous  function  x  €  C«[0.oc).  so  the 
continuous  mapping  principle  yields  the  following  theorem. 

Theorem  10.  Suppose  p  =  1  and  EUl  <  ».  <  00 •  Then 

n-lliW[nt]  =>  (*\+<rl)u,f(B)(t)  in  Dr[Q,<x>) 

The  mapping  /  appearing  in  (5.2)  is  called  a  “reflection  mapping”;  such  mappings  arise 
naturally  in  the  analysis  of  queues  in  heavy  traffic.  Thus,  the  following  result  for  Q(t)  D(t 
should  not  come  as  a  surprise. 

Theorem  11.  Suppose  p  =  1  and  fCnJ  <  oc  EVn 2  <  oc.  Then 

r»-I/JQ(rjf)  =>  <  A3<r^  *■  p3<r\)xn 
n:n  (  -  Xt)  =>  g(X3ff:ABx  .n3<TJsBt)(t) 


in  0*{O.oo),  where  B\,Bi  are  independent  real- valued  standard  Brownian  motions  and  ? 
0*i[O.  »)  —  OnfO.oo)  is  defined  by  g(z.  y)f <)  =  y(t)  *■  mf{x(»)  -  y(j)  Q  <  s  <  t} 

To  use  the  above  approximations,  one  needs  to  study  the  distribution  of  the  "reflected" 
process  f(8).  (Such  processes  are  also  known  as  “regulated  Brownian  motion.")  It  can 
be  shown  that  /(B)  =  |fl|  so  that  the  continuous  mapping  principle  yields  results  of  the 
following  kind:  if  eJ.ej  >0  and  p  =  i.  then 


p\w«  >  x(<r;  *.  (T5)t/3n!/I}  —  P{!5(1)|  >  xj  =  j  exp(-t3/2  )dt 

P  (  max  Q(t)  <  ( A3<r;  -  j:'  :xn;/:  1 

PS'S"  ~  j 

-  |S(t)i  -  *}  =  1  '  577-rexp{-[»J(2*  +  l):/8x3]}. 

k  1  t>i 

as  n  - 

The  question  also  arises  as  to  whether  Theorems  10  and  ll  are  true  diffusion  ap¬ 
proximations.  in  the  sense  that  the  limit  processes  are  diffusions.  In  particular,  is  /( 3  <  a 
diffusion  process?  The  path  continuity  of  [(B)  is  obvious.  To  see  that  f(B)  has  the  Markov 
property,  one  notes  that  if  x(0)  *  0,  then  [(t)  =  /(x)  where 

/(*)(<)  =  x(t )  -  min{:(i)  A  0  :  0  <  i  <  t). 

Since  fl(0)  =  0 ,f(B)  »  f(B).  FVirthermore,  if  t  >  u,  then  /(*)(<)  *  /(#.*)(«  -  u)  where  (0„x>i  jt  = 
x(j  +  u)  -  *(«)  +  /(*)(«)•  Thus,  the  distribution  of  f(B)(t),  conditional  on  f(B)(»)  for  s  <  u.  :s 
identical  to  the  distribution  of  /(«  +  B)(t  -  u)  where  x  =  f(B)(u)  (the  independent  increments 
property  of  8  was  used  here).  Thus,  the  reflection  functional  /  preserves  the  Markov 
property  in  this  setting. 

We  turn  now  to  the  statement  of  certain  variants  of  the  above  heavy  traffic  results, 
which  hold  for  stable  and  unstable  queues  in  which  p  »  l.  The  idea  here  is  to  consider  a 
sequence  of  CI/C/ l/cc  queueing  systems  in  which  the  arrival  and  service  distributions  '.-a:;, 
with  the  queue.  Suppose  that  in  the  n’th  system,  the  inter-arrival  and  service  time  r 
are  given  by  the  sequences  {t',;  ;  >  1}  and  {i',,  ;  >  o).  Set  a;1  =  El\,  =  £V„.  = 


var Un) , <r^  =  v&r  VB; ,  and  assume  that  0  <  A"1,^"1  <  co.  We  further  require  that: 


(5.3) 


i) 

ii) 

iii) 

iv) 

v) 

vi) 


An  -  A, 


'  An 


a 


2 

A' 


Pn  — 


there  exists  c  >  0  such  that 
supf  (|l^|J+*  +  |V„|J+‘)  <  oo 

n>0 

n*/J(A„  -  pn)  —  c, 


0  <  A  <  oo 
>0 

0  <  p  <  oo 
<r|  >  0 


-oo  <  e  <  oo. 


Just  as  in  the  case  of  the  ordinary  central  limit  theorem  for  r.v.’s,  there  exists  a  version  of 
Donsker's  theorem  that  governs  doubly  indexed  families  of  r.v.’s  (see  PROHOROV'  ( 1956)  i. 
To  be  precise,  let  e  :  [0,  oc)  —  R  be  given  by  e(t)  =  t  and  consider 


<n(0  =  "‘/J 


Then,  xn  =>  eAB  in  D*[0,oo);  a  similar  FCLT  holds  for  the  service  times.  By  applying 
continuous  mapping  ideas  similar  to  those  used  in  the  p  =  1  setting,  we  obtain  the  following 
limit  theorem.  ( Wn ,  Qn( •),  Dn(  )  are  the  waiting  time,  queue-length,  and  departure  processes 
associated  with  the  n’th  queue). 


Theorem  12.  Assume  (5.3)  i)-vi). 

a)  If  -do  <  c  <  o c,  then 

n~l,,Qn(nt)  »  f{(\3<r\+p3<r\)ll3B  +  ct)(t) 
n-l/1  (**)  g(X3<r\Bi  +ce,p3<r3sBi)(t) 

in  Dr[0,oo),  where  Bi,Bt  are  independent  standard  Brownian  motions. 

b)  If  c  =  oo,  then 

(«^2£! .  ,A„ .  „)  »  ,av;  +  AJ)'"B«) 

,i/j  (£21211  -fi't'J  »  v3,,ffSB(t) 


in  D*[0.  x). 

c )  If  c  =  -x.  then 


n-l/3Qn(nt)  =»  0 


1/2 


r>n'nt) 


-  An;j  =>  A 3/7ffAB(t) 
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in  Dr[0,oo) 


A  similar  result  holds  for  the  waiting  times. 

Theorem  13.  Assume  (5.3)  i)-v)  and  replace  vi)  by  -  A"1)  —  c,  -oc  <  c  <  x.  Then 

n-l/iW('nt]=>  f((<Tl+<T2s)ll7B  +  ce)(t)  if  -  oc  <  e  <  oc 

/  wn  \ 

„-l/2  =>(^+Or|)I/>S(0  if  C=00 

n_1/J W^(]  ^  0  if  c  =  -oc. 

For  further  details  on  the  above  heavy-traffic  limit  theorems  in  which  p  «  1.  see  IGLL- 
HART  and  WHITT  (1970  a.b).  To  fully  utilize  the  above  approximations,  one  needs  to  be 
able  to  analyze  the  resulting  limit  processes.  One  complication  that  arises  in  the  setting 
of  Theorems  12  and  13  is  that,  unlike  the  p  =  1  situation  in  which  f{B)  1  |B|,  it  is  not  true 
that  f(aB  +  ce)  =  |qS  +  ce|.  However,  it  turns  out  that  one  can  develop,  using  methods  to 
be  discussed  in  Section  6,  an  analytical  theory  for  the  process  /(a£  +  ee);  this  process  is 
known  as  reflected  (or  regulated)  Brownian  motion  with  drift  c  and  variance  parameter  a 
Among  the  results  available  is  the  distribution  of  f{aB  +  ce):  for  o  >  0, 

(5.4)  P{f(aB  +  ce)(t)  <  x}  =  *  ( -  «p(2cx/a2)*  (^T7T ) 

where  #(x)  =  (2*r1/3  /*„  exp(-t3/2)A  is  the  standard  normal  distribution  function.  We  will 
now  illustrate  the  application  of  this  result  to  a  CI/G/ l/oo  queueing  system. 

Example  6.  Consider  the  analysis  of  {Wn  :  n  >  0}  for  a  GI/G/ l/oo  queue  in  which  A  = 
Suppose  that  we  wish  to  study  the  distribution  of  Wn  for  n  large,  using  (5.4).  The  idea  is 
to  set  a2  =  +  c  =  -  A-1).  Then,  Theorem  13  yields 

(5  5)  P{Wn  <  xnl/J}  as  P{f(W\  +  <r\)U7B  +  [nl/V‘l  -  A"l)]e)U)  <  *) 

or.  more  descriptively, 

Wn  Z  nU7f((<r\  +  <t| )uiB  +  [nl/J(ii_I  -  A-1 )]«)( 1). 

Note  that  the  right-hand  side  of  (5.5)  can  be  evaluated  using  (5.4). 

In  Example  6.  observe  that  if  c  =  -  A'1)  is  a  large  negative  (large  positive 

number  (this  occurs  if  n  is  large  relative  to  -  A-1)),  the  approximation  >  55  ■  .?  ?'-■"■ 
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since  the  r.v.  f(aB  +  «)(1)  is  then  essentially  a  point  mass  at  zero  (infinity)  Better 
approximations  are  available  if  c„  =  n1/J{^-1  -  A-1)  is  large,  however.  Relation  (5.5)  says 
that  Wn/n1**  Z  f_(aB  +  cne)(  1).  Proceeding  formally,  this  suggests  that 

(m_1  -  A ~l)Wn  =  CnWJn1'1  Z  enf(aB  +  c„e)(l). 

But  the  r.v.  cnf(aB  +  cne)(l)  is  equivalent  to 

f(acnB  +  c*e)(l)  2  f(aB(d)  +  t(c\  ))(1)  =  f(aB  +  e)(c3n) 

(the  first  equality  relies  on  the  scaling  behavior  of  Brownian  motion:  yB(  )  =  B(y2  )).  Note 
that  if  (fi~l  -  A-*)  is  negative  and  c„  is  large,  this  argument  yields  the  approximation 
|p_1-A_1| Wn  Z  f(aB-e)(o c).  Turning  to  (5.4)  and  letting  t  —  oo  it  turns  out  that  f(aB-e)(t j  => 
aJexp(l)/2  as  t  —  oo.  where  exp(l)  is  an  exponential  r.v.  with  parameter  one.  Thus,  we 
approximate  \n~l  -  A_l|Wn  via  aJexp(l)/2.  If,  on  the  other  hand,  (p-1  -  A-1)  is  positive  and 
e„  is  large,  |/i“l  -  A — 1 1 Wn  a  f(aB  +•  e)(c2).  Because  of  the  positive  drift  of  aB  +  e,f(aB  ■+■ 
«)(<£)  Z  (aB  +  t)(cl)  2  cn(aB(  1)  +  c„e)( D  i.e.  Wn/n1'2  -  c„e(l)  =  nl'*(W„/n  -  (n~l  ~  A-1))  Z  aB(  1). 
We  summarize  the  above  discussion  with  the  following  table  of  approximations: 


Parameter  Region 

Approximating  Distribution 
for  Wn 

i 

(a*~  1  -  A-1)  small 

n1/J(p-1  -  A-1)  large  and  negative 

igte^iyexpd) 

n  large,  (p~l  -  A-1)  small, 

-  A*1)  small  to  moderate 

nl/,/((*i  +  *1 )lliB  +  ((m“1  -  A->)n‘/J]e)(l) 

(/4~l  -  A~l)  small 

nl,2(n~l  -  A-1)  large  and  positive 

n(ii~l  -  A-1)  +  nx^(ff\  +  <r|)1/2S(  1) 

Except  for  the  topmost  entry  in  the  table,  the  approximating  distributions  cited  above 
are  given  by  Theorem  13.  For  the  entry  concerning  the  situation  in  which 
is  large  and  negative,  PROHOROV  (1963)  proved  the  appropriate  result  (see  KINGMAN 
1 1961 )  for  a  related  theorem). 
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Theorem  14.  Assume  (5.3)  i)-v)  and  suppose  that  nlf7(n~l  -  a~1)  —  -x.  Then 
I -  A- MW,  ^  (<r\  +  <r|)exp(l)/2  as  n  —  oo. 

Theorem  14  suggests  that  a s  p  /  l.EW,  ~  A(ffJ  +  »|)/2(l-p)  where  W,  is  the  steady-state 
waiting  time  associated  with  the  p’th  queue.  In  fact,  this  heavy-traffic  approximation  is 
an  upper  bound  on  EW  for  all  p;  see  MARSHALL  (1968). 


Theorem  15.  In  a  Gl/G/  1/oc  queue  with  p  <  1  ,EW  <  +  *j)/2(l  -  p). 


To  conclude  this  section,  we  note  that  further  details  on  the  structure  of  reflected 
Brownian  motion  /(aB  +  c<)  may  be  found  in  ABATE  and  WHITT  ( 1987a, b).  It  also  turns 
out  that  the  results  of  this  section  are  valid  under  much  more  general  hypotheses  than 
those  stated  above.  In  particular,  the  results  hold  even  when  the  inter-arrival  and  service 
time  sequences  are  dependent.  This  is  important,  since  typically  the  arrival  stream  to  a 
queue  is  the  superposition  of  departure  processes  from  other  queues;  such  an  arrival  stream 
yields  dependent  inter-arrival  times.  To  deal  with  dependent  sequences,  it  is  enough  to 
observe  that  the  above  results  essentially  require  only  that  the  arrival  and  service  streams 
satisfy  a  joint  FCLT:  There  exists  A_1,/j*‘  and  £  such  that 


(5-6) 


=>E1/JB(t) 


in  £>*i[0,oo).  Using  (5.6),  analogues  to  Theorems  7,  8,  9,  10,  and  11  may  be  obtained. 
Analogues  to  Theorems  12,  13,  and  14  depend  on  assuming  a  double-indexed  version  of 


(5.6). 

Finally,  ROSENKRANTZ  (1978)  has  studied  the  question  of  how  rapidly  the  distri¬ 
bution  of  Wn/n117  converges  to  that  of  reflected  Brownian  motion,  when  the  queue  is  in 
heavy-traffic  (p  =  1).  By  using  the  strong  approximation  theorem  of  Section  4,  he  shows 
that  the  rate  is  0(logn/n1/3). 


6.  Background  on  Diffusion  Processes 

In  this  section,  we  briefly  describe  several  important  elements  of  the  general  theory  for 
diffusion  processes.  We  are  particularly  interested  here  in  describing  how  the  theory  of  dit- 
fusion  processes  is  intimately  connected  to  the  study  of  certain  partial  differential  equations 
(PDE's).  It  is  that  connection  to  PDE's  that  makes  diffusion  processes  computatior.a..;- 
and  analytically  attractive. 
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To  develop  the  connection  with  PDE’s,  we  review  briefly  the  different  modeling  ap¬ 
proaches  that  give  rise  to  diffusion  processes: 

1. )  A  diffusion  process  may  be  directly  postulated  as  am  appropriate  model  (e.g.  Brownian 

motion  as  a  model  of  molecular  motion). 

2. )  A  diffusion  process  may  arise  as  a  limit  occurring  in  the  study  of  a  stochastic  model 

(e.g.  reflected  Brownian  motion  arises  as  a  limit  in  the  study  of  queues  in  heavy  traffic). 

3. )  One  may  postulate  that  the  stochastic  process  X  driving  a  model  has  certain  in¬ 

finitesimal  characteristics.  Specifically,  if  X  is  f^-valued,  one  specifies  functions  n 
R4  —  R4,<r2  :  R4  —  R4xd  called  the  infinitesimal  drift  and  infinitesimal  covari¬ 
ance  functions,  respectively.  These  functions  p,<r2  are  related  to  X  by  requiring  that 
E{X{t  +  A)|X(tt)  :  u  <  t)  =  n(X(t))h  +  o(h),  cov{.Y(<  +  A)|X(u)  :  u  <  f}  =  <r2(X(t))h  +  o{h).  If 
e2(  )  is  positive  definite  and  X  is  additionally  postulated  to  be  Markov  with  continuous 
paths,  then  the  resulting  process  (if  it  exists)  must  be  a  diffusion  process. 


Much  of  the  theory  of  diffusion  processes  concerns  mathematical  difficulties  that  arise  when 
a  diffusion  process  is  postulated  as  in  3.)  above.  More  precisely,  given  and  <rJ,  does  there 
exist  a  unique  diffusion  process  X  (unique  in  distribution)  for  which  E{X(t  +  A)|A(u)  u  < 
<}  =  n(X(t))h  +  o(h),co\{X(t  +  /»)|A(ti)  v  <  t]  =  <r3(X(t))h  +  o(h)?  To  answer  this  question, 
suppose  there  exists  such  a  process  X.  If  /  is  sufficiently  smooth, 


E'HX(h))  =  E, 


d*f 

dz  tdzj 


(0(W) -*.•)(*>(*)-»,) 


where  f  is  on  the  line  segment  joining  x  and  X{h).  Using  the  infinitesimal  characteristics, 
the  above  Taylor  expansion  yields 


(61) 


E'f(X(h))  =  f(x)  +  (Lf)(z)h  +  o(h) 


where 


(£/)(*)  = 


ISl 


».;  =  l 


This  expansion  holds  for  a  large  class  of  /’ s.  call  it  V{L).  One  can  state  (6.1)  differently 
Let  P,(z. dy)  =  Pr{.Y(t)  €  dy)  be  the  transition  measure  for  X.  Then.  (6.1)  asserts  that 


(6.2) 


h~'‘  J  P*iz.dy)f{y)  -  f(x) 


—  (Lf){x) 


as  h  i  0,  provided  /  €  I>(I).  Of  course,  the  family  (P,  .  t  >  0)  must  have  the  semigroup 
property 

(6.3)  P, +,(*.*)  =  j  Pt(*'dy)P,(V,A) 

and  it  must  be  stochastic:  Pt  >  0,  P,(x,  K4)  =  1.  An  extensive  theory  has  been  developed  to 
determine  when  there  exists  a  stochastic  semigroup  satisfying  (6.2),  for  a  given  L  and  V(L). 
(A  principal  result  here  is  the  Hille-Yosida  theorem;  see  KARLIN  and  TAYLOR  (1981)).  A 
positive  answer  to  this  question  generally  guarantees  the  existence  of  a  diffusion  X  having 
infinitesimals  p  and  <rJ.  (One  also  needs  to  check  that  a  version  of  X  can  be  constructed 
with  continuous  paths,  and  thus  existence  of  ( P, )  may  not  be  quite  enough  to  guarantee 
existence  of  a  diffusion.) 

Suppose  now  that  we  are  given  that  a  diffusion  .V  exists,  described  by  L  and  V(L).  The 
operator  L  is  called  the  infinitesimal  generator  of  .V  and  V{L)  is  called  the  domain  of 
the  generator.  We  shall  now  describe  some  of  the  problems  related  to  X  that  may  be 
found  by  solving  PDE’s  involving  L. 

Problem  1.  Let  A  be  a  real-valued  diffusion.  Let  T(a)  =  »n/{t  >  0  :  X(t)  =  a}  be  the 
first  time  that  X  takes  on  the  value  a.  For  fixed  levels  a  and  5(a  <  6),  let  u(r)  =  Pt{T(a)  < 
T(i)}(a  <  x  <  b)  be  the  probability  that  X  hits  level  b  before  hitting  level  a.  Then.  ucV(L) 
and  u  satisfies  the  differential  equation 


Lu  =  0, 

subject  to  the  (obvious)  boundary  conditions  u(o)  =  1,  u(6)  =  0. 

Example  6.  If  X  is  a  one- dimensional  standard  Brownian  motion,  it  is  easily  verified  from 
the  stationary  independent  increments  of  X  that  p(x)  =  0  and  <rJ(x)  =  1.  The  domain  V(L) 
consists  of  all  bounded  functions  /  :  R  —  #?  having  a  bounded  continuous  second  derivative 
The  solution  to  the  differential  equation 

subject  to  u(a)  =  1.  u(6)  =  0  is  the  affine  function  u(z)  =  (6  -  z)/(b  -  a). 

Problem  2.  Let  i  be  an  interior  point  of  a  set  .4  with  a  “nice"  boundary  and  assume  that 
Pv(r(.4')  <  x}  =  1  for  all  y  €  .4.  where  T(. 4')  =  mf{T  >  0  :  ,Y(t)  €  .4'}.  For  given  real-valued 


functions  g  and  k ,  set  u>(x)  =  Et  {/0r(/4  )e*P(-  ft  k(X(r))dr)g(X(t))dty  Then,  w  satisfies  the 


PDE 


(Lw)()-k()w()  =  -g()  in  A 


subject  to  w  e  V(L)  and  u>(y)  =  0  on  the  boundary  of  A. 

Example  6  (continued).  Let  k  =  0  and  g  =  l  in  Problem  2,  so  that  u>(z)  =  EXT(A')  is  the 
expected  amount  of  time  required  for  the  diffusion  X  to  “escape”  from  the  set  A.  If  .V  is  a 
one-dimensional  standard  Brownian  motion  and  A  =  [a,fc],  then  w  is  obtained  by  solving 

1  . 

subject  to  w(a)  =  u(6)  =  0.  The  solution  is  u-fr)  =  ( z  -  a)(b  -  z). 

Problem  3.  For  a  given  real-valued  g,  set  u(t.z)  =  Etg(X(t)).  Then,  u  satisfies 

(64)  lu(t  2)  =  +  \  y,  9l^d^STiu{t'x) 

subject  to  u(t,  )  e  V(L)(  and  u(0,  )  =  g(  )\  in  short-hand,  (6.4)  is  written  u«  *  Lu.  Equation 
(6.4)  is  called  the  backward  equation  for  X,  and  is  fundamental  to  the  study  of  diffusion 
processes.  Assume  that  P(t,z,dy)  has  a  density  p(t,x,y)  with  respect  to  Lebesgue  measure 
dy.  By  formally  setting  g(-)  =  6y(  )(6,(  )  is  the  Dirac  delta  function),  (6.4)  yields  an  equation 
for  the  transition  density  p : 

(6  5)  = 

•  Si  IjSl  3 

subject  to  p(t,  ,y)  €  T>{1)  and  p(0,-,y)  =  6y( •);  the  partial  differential  equation  (6.5)  can  be 
re-written  in  the  more  convenient  shorthand  notation 

d  . 

ap  = Lp 

If  ,Y  is  a  process  that  lives  on  all  of  R*  (so  that  .Y  has  no  non-trivial  “boundary  behavior  " 
one  can  obtain  the  “adjoint"  equation  to  (6.5): 

(6  6)  1^  ^)  =  +  5  E  d^r,(tr't{y)p{t'z'y)) 

i  =  l  *'  i ;  =  1  '  ** 

subject  to  p(0.z.  )  =  fr(  ):  (6.6)  is  the  forward  equation  for  the  density  p.  The  adjoint 
equation  can  be  derived  by  viewing  the  "backwards  operator"  L  as  an  operator  on  ar. 


appropriately  chosen  space  of  smooth  functions,  and  performing  an  “integration  by  parts" 
to  obtain  the  adjoint  (or  forward)  operator  L’ . 


Example  7.  Let  X  be  a  real- valued  diffusion  process  for  which  m(z)  =  -pz  and  <rJ(x)  =  1: 
this  is  a  special  case  of  a  one  dimensional  Ornstein-Uhienbeck  process.  (Note  that  if 
fi  =  0,  X  is  a  standard  Brownian  motion.)  Then,  the  backwards  equation  for  the  transition 
density  p  takes  the  form 

h  i  dp 

at  ~  2<9*J  dx 

subject  to  p(0,*,y)  =  6y(z).  On  the  other  hand,  the  forward  equation  is  given  by 


dp 

dt 


1  d2p 
2W 


subject  to  p(0,*,y)  =  Sy(z j.  If  p  =  0,  the  (common)  solution  p  is  given  by 


p(t,z,y)  =  *{t,z,y) 


where  p  is  the  Gaussian  kernel 

*>(»■  *.  y)  =  (2i exp(-(y  -  x)*/2t) 
t  >  0).  However,  if  p  yt  0,  the  solution  p  takes  the  form 

p(t,x,y)  =  p  ^(1  -e-2‘“),xe-'‘',yj 

Problem  4.  For  a  real- valued  g  and  a  non-negative  function  k,  consider  the  expectation 

u>(t,  *)  =  Ez  | exp  j  k{X(s))dt  j  $(*(0)  j  . 

Then,  under  suitable  regularity  conditions  on  X,  g ,  and  k,  it  can  be  shown  that  w  solves 
the  PDE 

S-w  =  Lw  -  kw. 
at 

subject  to  u’(<.  )cV{L)  and  u/(0,  •)  =  y(  ). 

Example  6  (continued).  Let  X  be  a  one- dimensional  standard  Brownian  motion  and 
suppose  that  i(x)  =  a  and  g(z)  =  z.  Then. 
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u;(t,x)  =  xe~at 


solves  Problem  4.' 

Problem  5.  Suppose  that  g  is  a  real-valued  function.  If  we  view  $(z)  as  the  rate  at  which 
cost  accumulates  in  state  z,  then 

u(*)  =  Er  r  ‘~a,g(X(t))dt 
Jo 

is  the  a-discounted  cost  associated  with  starting  the  diffusion  X  in  state  z.  Under  suitable 
regularity  conditions,  one  can  show  that  ur T>(L)  and  satisfies 

at i  -  Lu  =  g. 

Example  6  (continued).  If  X  is  a  one-dimensional  standard  Brownian  motion  and  a  >  0, 
then  the  solution  u  to  Problem  5  is  given  by 

1  eOO 

“(*)  =  J  exp  (-v^lz-y|)  g{y)dy 
(provided  the  integral  exists  and  is  finite). 

Problem  6.  Suppose  that  X  is  positive  recurrent  so  that  a  density  p(  )  exists  such  that 
p{t,x,y)  —  p(y)  as  t  —  o©  for  each  z.  This  type  of  limit  behavior  suggests  that  dp(t,z,  y)/dt  —  0 
as  t  —  oo;  formal  substitution  in  (6.6)  yields 

(67)  \  ,£  d£dii{ff'>iy)p{y)) "  S  l‘(M’(y)p(v))  =  ° 

subject  to  / p(y)dy  —  1;  (6.7)  can  be  solved  to  obtain  the  steady-state  density  of  X.  (We 
caution  that  if  X  has  non-trivial  boundary  behavior  (e.g.  reflection),  additional  boundary 
conditions  must  be  prescribed.)  Typically,  the  process  X  becomes  a  stationary  process 
when  initialized  according  to  the  density  p.  Hence,  p  is  often  termed  a  stationary  density 
for  /V. 

Example  7  (continued).  Suppose  that  .V  is  a  one-dimensional  Omstein-Uhlenbeck  process 
with  p(z)  =  -pz  and  <rJ(z)  =  1.  Then,  the  stationary  density  p  must  satisfy 

1  <P  d 

:-TPlyl  —inyp(y))  =  0. 

2  dy  ay 
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p  exists;  it  is  given  by 


end  TAYLOR  (1«1). 


“MS*)- 


I  the  analytical  th^cy  for  diffusions  is  given  in  KARLIN 


In  meet  operations  research  applications,  diffusions  a»  not  obtained  vU  an  infinitesimal 
Aaractdiaalionof  the  process;  rather,  as  illustrated  in  Section  5,  they  are  usually  obtained 
as  Kant  processes  that  are  functionals  of  Brownian  motion.  In  order  to  use  the  PDE’s  cited 
above,  one  needs  to  calculate  £  for  such  processes.  We  n ow  indicate  hoar  to  do  this  for 
reflectedBrownian  motion  f(aB  ♦  «);  the  tod  used  is,  however,  quite  general 

Let  X  be  a  reflected  Brownian  motion  starting  at  *  >  0.  Then,  for  t  >  0,  X (t)  * 
j{aB  +  *  +  «)(t)»  where  /(»)(<)  *  y(i)  -  mia{y(«)  A  0 : 0  £  »  £  t}.  Hence,  we  may  represent  X  as 
•*(*)*  +  V(l),  arhere  V(t)  *min{(*+oB(j)  +  c#)A0:0  <«<<}.  The  process  V  has 

two  ^aportaoC  characteristics.  Firstly,  V  is  non-decreasing;  hence,  it  may  be  used  as  ad 
integrator.  Secondly,  the  points  of  increase  occur  only  at  those  *  for  which  X(t)  vanishes 
(i.e.,  £**(*<•)  >0)V<*)*0). 

We  are  going  to  use  ltd’s  formula  (see  the  Markov-  process  chapter  of  this  handbook )  to 
dsrive  the  backward  equation  for  the  diffusion  X.  Consider  afonction  dM)  oa^,T]x  fo.oc) 
which  is  twice  continuously  differentiable  mi  its  domain.  We  will  apply  an  It6- type  formula 
to  the  process  v(t,X(t)),  where  *(!,*)  *  q(T * «,»).  Note  that  the  stochastic  differential  of 
the  process  v(t,X(t)\  is  given  by 

H*,XW  m  ^(*rX(t)m  +  J|*(t,*(t))dX(l) 

+  5  !&(*’Xm4X(t))*. 

Clearly,  dX(t)  m  odB(t)  +  ed  -  dV(t).  On  the  other  hand,  the  ltd  calculus  asserts  that 
(4X(i))*»o>«.  Bence, 

(«S)  A<l,X«))«{^(i.X(l))  +  c^(i.X(t))  +  y^[(*,X(t))}dl 

+  o£(t,X(t))dB(t)  -  £(t.X(t))dV(t). 

Suppose  now  that  we  chooee  the  function  9  so  that  the  bracketed  term  in  (6.8)  disap¬ 
pears,  i.e.,  choose  17  to  satisfy  the  PDE 
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We  can  simplify  (6.8)  further  by  observing  that  since  V  increases  only  when  X  is  zero,  we 
have 

(6.10)  ^(t,X(t))dV(t)=j^(t,0)dV(t). 

Hence,  (6.10)  vanishes  if  we  require  that  r;  satisfy  the  boundary  condition 

(6.11)  Aff(t,0)  =  0. 

We  may  therefore  conclude  that  if  satisfies  (6.9)  and  (6.11),  then 

di/(i,A’(0)  =  o^(<.A'(0)dfl(0 

and  thus 

i/(T.  X(T))  -  i/(0,  - V ( 0 ) )  =  a  t.X(t))dB(t ). 

But  the  right-hand  side  is  a  martingale  (see  the  Markov  process  chapter  of  this  handbook) 
so  we  conclude  that 

£MT,X(T))  =  v(0,x) 


i.e., 

EtT,(0'X(T))  =  r,(T,z). 

If  we  set  j(x)  =  »j(0,  x),  we  find  that  f?(T;z)  =  Etg(X(T)),  provided  that  rj  satisfies  (6.9)  and 

(6.11). 

A  glance  at  (6.4)  shows  that  rj  is  the  solution  to  the  backwards  equation  corresponding 
to  the  diffusion  X  -  f(aB  +  ee).  Thus,  the  infinitesimal  generator  L  of  X  is  given  by  the 
second-order  differential  operator 

,  d 

dz  +  2  dr3’ 

the  domain  V{L)  of  the  reflected  Brownian  motion  X  includes  bounded  functions  h,  that 
possess  a  bounded  continuous  second  derivative  and  satisfy  d/dz  )>(0)  =  0. 

Hence,  Ito's  formula  can  be  applied  to  determine  both  the  infinitesimal  generator  and 
the  “boundary  conditions”  that  characterize  a  diffusion  process. 


7.  Diffusion  Approximations  for  an  Open  Network  of  Queues  in  Heavy  Traffic 

In  Section  5.  we  showed  how  a  single-server  queue  in  heavy  traffic  can  be  approximated 
by  a  one-dimensional  reflecting  Brownian  motion.  In  this  section,  we  discuss  how  v 
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approximate  aa  open  network  of  d  queueing  stations  in  heavy  traffic  by  a  rf-dimensional 
diffusion  process. 

Consider  a  network  of  d  queueing  stations.  Each  of  the  d  stations  consists  of  a  single 
work-conserving  server  that  serves  customers  in  the  order  in  which  they  arrive.  By  work 
conserving,  we  mean  that  the  server  never  goes  idle  when  facing  customers  in  its  queue.  We 
assume  that  the  customer  routing  within  the  network  is  Markovian.  Let  P  be  the  routing 
matrix  for  the  network  (i.e.,  P,j  represents  the  probability  that  a  customer  completing 
service  at  station  «  goes  immediately  to  station  j;  1  -  Pij  is  the  probability  that  a 
customer  released  from  station  « leaves  the  network).  To  simplify  our  following  discussion, 
we  assume  that  Pti  -  0  for  1  <  •  <  d.  We  further  require  that  P  be  irreducible  and  that 
(/-  P)_1  exist  (i.e.,  P  is  substochastic). 

Let  U,  =  {U,(m)  :  m  >  1}  be  the  sequence  of  exogenous  customer  arrivals  to  the  i'th 
station;  the  U,(m)' s  are  assumed  to  be  i.i.d.  with  mean  l/Aj  and  (finite)  coefficient  of 
variation  a,J  (i.e.,  vuUi(m)  =  \~3a*).  For  l  <  •  <  d,  let  K  =  { V; (m)  :  m  >  1}  be  a  sequence 
of  i.i.d.  unit  mean  r.v.’s  for  which  6?  =  vart;(m)  <  oo;  the  r.v.  v;(m)  can  be  viewed  as  the 
service  requirement  of  the  m’th  customer  to  be  served  at  station  i.  If  m(n)  is  the  service 
rate  at  the  j’th  station  in  the  n’th  approximating  network,  then  Vj(m)ln,(n)  is  the  actual 
duration  of  service  for  the  m’th  customer  at  the  >’th  station  in  the  n’th  network. 

Let  Q„(0)  be  a  random  d- vector  having  non-negative  integer- valued  components.  The 
i’th  component  is  to  be  interpreted  as  the  number  of  customers  that  are  sitting  in  the 
queue  of  the  i’th  station  waiting  to  be  served  at  time  t  =  0  (in  the  n’th  system). 

We  require  that  the  2d  inter-arrival  and  service  requirement  sequences,  as  well  as  the 
customer  routing  dynamics  and  Q„( 0),  be  mutually  independent.  Such  a  queueing  network 
as  we  have  described  is  termed  a  generalized  Jackson  network.  (Note  that  if  the  r,(m)  s 
and  V,(m)’s  are  exponential  r.v.’s,  we  obtain  a  standard  Jackson  network.) 

The  theory  of  weak  convergence  for  queueing  networks  is  largely  based  on  the  devel¬ 
opment  of  convenient  representations  for  the  corresponding  queue-length  processes.  By 
applying  the  continuous  mapping  principle,  random  time  change  theory,  and  converging- 
together  techniques  to  the  representation,  one  arrives  at  a  diffusion  limit  for  the  network. 
To  describe  the  representation  that  is  useful  here,  we  let 

A,(t)  =  max  |  m  >  0  :  ^  i\(j)  £  < 
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be  the  process  which  counts  the  number  of  exogenous  arrivals  to  the  i’th  station  over  the 
interval  (0,tj.  We  further  let 


C,(t)  =  max 


note  that  C,,(t)  =  C,(p,(n)t)  is  a  process  which  counts  the  number  of  customers  served  at 
the  *’th  station  during  the  first  t  units  of  busy  time  (in  the  n’th  network).  Let  <?*(r)  = 
(Q«i(0,  .<?»*(<))  be  the  vector  queue-length  process  in  which  Q„,(t)  represents  the  number 

of  customers  at  the  i’th  station  at  time  t  in  the  n’th  approximating  network.  Note  that 


B«.(0 


/«?„,(*)  >  0)* 


is  the  amount  of  time  that  the  i’th  station  is  busy  over  [0, <].  Then,  Cn,(Bn,(i))  is  the  total 
number  of  customers  served  at  the  t’th  station  (in  the  n’th  system)  over  [O.t]. 

Suppose  that  R,(m)  is  a  random  vector  which  routes  the  m’th  customer  completing 
service  at  the  i’th  station.  More  precisely,  R,(m)  =  e,  (e,  is  the  >’th  unit  vector)  if  and 
only  if  the  m’th  customer  to  be  served  at  the  i’th  station  is  routed  immediately,  upon 
completing  service  at  station  i.  to  station  j.  Then, 

C*,(  £«•(<)) 

£  *.(» 

;  =  i 

is  a  vector  in  which  the  i’th  component  equals  the  number  of  customers  routed  from 
station  i  to  k  over  [0, <].  Let  Cn(B„(t))  =  (C„i(flni(i)),C*j(0nj(t)). •  •  •  ,C„i(Bnd(t))  and  .4(0  = 
(/MO - 4«(0).  Clearly, 

i  r)) 

(7.2)  Q0(o  =  g*(o)  +  4(o  +  £  £  «.(»-c„(fln(0)- 

13 1  ;  =  1 

By  arguing  path-by-path,  it  is  straightforward  to  show  that  the  queue-length  process  Qn 
(together  with  the  busy  time  processes  Bn i...  .,Bni)  is  the  unique  solution  of  the  coupled 
system  (7.1)-(7.2).  However,  we  shall  shortly  describe  an  alternative  representation  for  Q n 
that  is  more  convenient  in  terms  of  implementing  weak  convergence  arguments  of  the  type 
mentioned  earlier  in  this  chapter. 

Before  proceeding  further,  we  need  to  derive  conditions  on  the  network  which  force 
each  of  the  stations  into  heavy  traffic.  We  let  X  =  (Al,...,A<)  be  the  vector  of  exogenous 
arrival  rates  and  n(n)  =  '>i(n).  .m{n))  be  the  vector  of  service  rates.  Then.  Ai  /  -  ?  '  > 
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the  vector  of  “effective  arrival  rates  ’,  in  which  the  »’th  component  is  the  rate  at  which 
customers  arrive  to  station  i  (from  both  within  and  outside  the  network).  Suppose  that 

(7.3) 

as  n  —  oo,  where  the  vector  ^  is  such  that  n ,  =  (A(7-  P)~l)i  for  1  <  i  <  d.  Hence,  each  station 
is  effectively  experiencing  a  traffic  intensity  of  1,  and  is  therefore  in  heavy  traffic. 

The  alternative  representation  for  Qn  is  obtained  by  first  centering  the  cumulative 
processes  appearing  in  (7.2).  The  centering  constants  are  suggested  by  (7.3)  and  the 
following  FCLT’s: 

(7.4)  -  X,t )  =>  Bu(\,aU) 


=>  B2l(6?t) 

n1/2  =>  A,l/Jfl,(t) 


where  a,  =  (A ,(j,k)  .  1  <  j,k  <  d)  and  a ,(;,*)  =  -P,}P,k(j  t  *)  and  a ,(>,;)  =  P,,(l  -  P,,)-  FCLT  s 
(7.4)  and  (7.5)  are  just  the  FCLT's  for  renewal  processes  described  in  Section  5.  whereas 

(7.6)  is  the  multivariate  version  of  Donsker's  theorem.  Let  (/i(n)fl«)(t)  =  (pi(n)Bni(0- 
M«<(n)Snd(t))-  Then, 

(7.7)  Qn(t)  =  Zn(t)  +  Yn{t)(l -P) 


where 


Zn(t)  =  Q„(0)  +  (A  +  rfn)P  -  ?(n))t  +  f n(t ) 


i  C..(B.,( «)) 


Mt)  =  (A(t)  -  At]  +  £  E  R,(k)-Cn(Bn(t))P 


i=l  k=l 


+  [Cn(Bn«))P  -(n(r>)Bn)(t)P] 
-[C,(fln(0)“(^n)fln)(0] 
y,(0  =  ftfnM  -  (#i(n)B,)(t). 

It  turns  out  that  the  process  V„  appearing  in  (7.7)  has  the  following  properties: 
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(7.8) 

i)  Yn  is  non-decreasing  (component-by-component)  with  V«(0)  =  0, 

ii)  Ynj(t)  increases  only  at  times  t  for  which  Qn,{t)  =  0  (i.e.,  /*  I(Qn,(t)  >  0 )Ynj(dt)  =  0). 

Furthermore,  the  pair  of  processes  (Qn,B„)  satisfies  (7.1)-(7.2)  if  and  only  if  (Qn,Yn) 
satisfies  (7.7)-(7.8).  Hence,  we  may  view  (7.7)-(7.8)  as  an  alternative  characterization  of 
Qn- 

Let  £>1^(0, oo)  =  {i£Dfj<[0,  oo)  :  x(0)  >  0}.  Then,  given  oo),  there  exists  a  unique  y 

such  that: 

(7.9) 

i)  q  =  z  +  y(/  -  P) 

ii)  y  is  non-decreasing  with  y(0)  =  0 

iii)  y,  increases  only  at  times  t  when  q,(t)  =  0.  1  <  j  <  d. 

Since  y  is  uniquely  defined  (for  each  ;),  we  may  write  y  =  h{z).  If  we  let  f(z)  =  z  +  h(z)(I- 
P),  we  can  rewrite  (7.7)-(7.8)  as  asserting  that  Qn  =  f(Zn).  The  representation  Qn  =  f(Zn) 
plays  a  central  role  in  the  heavy  traffic  analysis  of  open  networks.  The  function  h  is  known 
as  the  regulator  map  and  /(z)  is  termed  the  regulated  version  of  z.  These  mappings 
were  first  studied  in  HARRISON  and  REIMAN  (1981).  The  function  h  has  the  following 
properties: 

(710) 

i)  The  function  /  is  (suitably)  continuous, 

ii)  For  every  t  >  0,  the  restriction  of  f(z)  to  [0,  f]  depends  only  on  the  restriction  of  ;  to 

M, 

iii)  Let  T  >  0.  Define  i(t)  =  q(T)  +  z(T  +  t)  -  z(T),  =  f(z(T  +  <))•  Then,  f(z)  =  /(:). 

Given  (7.10)i,  weak  convergence  results  for  Q„  can  be  obtained  by  suitably  approx¬ 
imating  Zn.  An  important  (and  typically  difficult)  step  in  dealing  with  Z„  is  to  show 
that 

cm 

n 

in  OflfO.oc)  as  n  —  oc  (i.e..  each  server  is  asympoticallv  busy  100%  of  the  time).  With 
iT.ll)  in  hand.  ( 7.4 )-( 7.6),  in  conjunction  with  a  random  time  change  argument.  permits 
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one  to  show  that  if  there  exists  #(0)  such  that 

(7.12)  n-1/J<?n(0)=>Q(0), 
then 

(7.13)  Zn(t)  =  n~l'*Zn(nt)  =>  Q(0)  +  c(I  -  P)t  +  rt/JS(t) 

as  n  —  oo.  The  process  B  appearing  in  (7.13)  is  a  (/-dimensional  standard  Brownian  motion 
and  the  covariance  matrix  r  is  given  by 

i 

(7.14)  r ]k  —  +  \}a*]6jk  -  Hjb^Pjk  ~  M***/5*;  +  +  6? P/*]. 

t= l 

By  applying  the  continuous  mapping  principle  to  the  function  /  and  the  process  Zn.  one 
obtains  the  following  diffusion  approximation  for  open  queueing  networks  in  heavy  traffic; 
see  REIMAN  (1984)  and  CHEN  and  MANDELBAUM  (1988)  for  further  details. 

Theorem  10.  Assume  (7.3)  and  (7.12).  Let  Qn(<)  =;  Qn(nt)/^/n  and  set  Z{t)  =  Q{ 0)  +  c(I  - 
P)t  +  rl/*B(t).  Then 

Q"  =  f(Zn)  =*•  f{Z)  =  Q 

in  DK4 [0,  oo),  as  n  —  oc. 

To  use  the  approximation  suggested  by  Theorem  16  in  a  practical  setting,  we  need 
a  queueing  network  in  which  each  station  is  in  heavy  traffic.  By  this,  we  mean  that  the 
difference  |(A(/  -  P)-1),/^,  -  l|  ought  to  be  of  order  t  for  some  small  e  (1  <i  <d).  (Think  of 
r  as  n_1/J  in  our  limit  theorem.)  Then,  the  diffusion  limit  Q  describes  the  fluctuations  of 
order  1/e  experienced  by  the  queueing  network  over  time  scales  of  order  1/e2. 

As  in  Section  5,  the  reflected  Brownian  motion  Q  (also  known  as  regulated  Brownian 
motion)  turns  out  to  be  a  diffusion  process.  This  basically  is  a  consequence  of  properties 
( 7. 10 )i-ii  of  the  map  /,  together  with  the  independent  increments  of  B.  The  term  reflection 
is  used  because  the  process  Q  can  be  viewed  as  “reflecting”  in  the  direction  of  the  «‘th  row 
of  I  -  P  whenever  the  i'th  component  of  Q  is  zero. 

The  diffusion  limit  Q  inherits  much  of  the  qualitative  structure  of  the  queueing  network. 
For  example,  we  note  that  if  c  >  0.  then  (7.3)  guarantees  that  each  station  can  serve 
customers  i  slightly)  faster  than  they  arnve,  so  that  the  network  ought  to  be  stable.  The 
following  result,  due  to  HARRISON  and  WILLIAMS  (1987).  gives  the  diffusion  analogue. 
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Theorem  17.  The  diffusion  process  Q  has  a  stationary  probability  distribution  *•  if  and 
only  if  a  >  0  for  1  <  i  <  d.  Furthermore,  any  stationary  probability  distribution  for  Q  is 
necessarily  unique. 

Recall  that  if  the  inter-arrival  and  service  time  distributions  are  exponential,  the  queue¬ 
ing  network  studied  here  is  a  Jackson  network  and  the  stationary  distribution  (when  it 
exists)  is  known  to  be  of  product  form.  This  product  form  structure  also  manifests  itself  in 
the  diffusion  limit  Q.  We  say  that  a  probability  distribution  *  on  ft*  is  of  product  form 
if 

i 

*(dzi  x  x  dzd)  =  )dzi. 

■=i 

The  next  result  is  also  due  to  HARRISON  and  WILLIAMS  (1987). 

Theorem  18.  The  diffusion  process  Q  has  a  stationary  probability  distribution  *  of 
product  form  if  and  only  if  c,  >  0  for  1  <  i  <  d  and 

(7  is)  2r;*  =  -(P*;r**  +  R;tr„) 

for  j  ^  k.  Furthermore,  if  a  product  form  stationary  probability  distribution  *  exists. 

P<(*)  =  m  txp(-niz)l(z  >  0) 

where 

In  addition,  HARRISON  and  WILLIAMS  (1989)  show  that  the  notion  of  quasire¬ 
versibility  for  queueing  networks  extends,  in  a  natural  way,  to  the  Brownian  limit.  Roughly 
speaking,  a  Brownian  model  of  a  queueing  station  is  quasireversible  if  and  only  if  the  de¬ 
parture  process  has  the  same  distribution  as  the  arrival  process.  It  is  believed  that  a 
network  of  quasireversible  Brownian  queueing  stations  must  necessarily  have  a  stationary 
distribution  of  product  form. 

We  note  that  in  the  Jackson  network  setting,  a2  =  62  =  1  and  (7.15)  is  easily  verified 
from  (7.14).  In  general,  the  stationary  probability  distribution  *  cannot  be  calculated 
in  closed  form  analytically.  Nevertheless,  the  Ito-type  argument  of  Section  6  (used  there 
to  justify  the  backwards  equation  for  one- dimensional  reflected  Brownian  motion)  can  be 
used  to  obtain  an  equation  that  typically  characterizes  the  stationary  distribution  *.  In 
preparation  for  the  statement  of  the  result,  let  F*  =  {zcR4  :  x*  =  0}  be  the  k'th  face  of  tr.e 

39 


non- negative  J-dimensional  orthant.  and  let  L  and  D  be  the  differential  operators  defined 
by 


and 


where  9  =  c(I  -  P).  Finally,  let  C2  be  the  space  of  functions  /  :  B*  — •  R  such  that  /  is 
twice  continuously  differentiable  and  such  that  all  partials  up  to  order  2  are  bounded  on 


{x  :  Xi  >  0, 1  <  i  <  d). 


Theorem  19.  Suppose  *■  is  a  stationary  probability  distribution  for  Q.  Then,  there  exist 
finite  measures  . ud  on  Fx _ Fd  such  that  for  each  feC 2, 


(7.16) 


/  (!/)(,),(*) +  i£  /  (D</)(iMdx)  =  0. 

2  Jr. 


See  HARRISON  and  WILLIAMS  (1987)  for  details. 

The  equation  (7.16)  is  known  as  the  basic  adjoint  relationship  satisfied  by  *.  Given 
the  importance  of  the  stationary  distribution  r  from  an  application  viewpoint,  significant 
activity  is  currently  underway  to  solve  (7.16)  numerically.  A  recent  paper  by  HARRISON 
and  NGUYEN  (1989)  specializes  its  numerical  treatment  to  d  =  2.  It  shows  that  in  the 
two  station  setting,  the  numerical  approximations  obtained  from  Theorem  19  give  good 
results,  in  comparison  with  simulation,  even  for  moderate  traffic  intensities. 

For  the  class  of  open  queueing  networks  that  we  have  described  above,  it  turns  out  that 
sojourn  times  can  also  be  approximated  (in  distribution)  by  appropriate  diffusion  limits. 

To  describe  the  notion  of  sojourn  time,  let  A  =  (Ax . hd)  be  a  rf-vector  having  non-negative 

integer-valued  co-ordinates,  and  fix  a  station  j.  (Let  us  adopt  the  convention  that  station 
0  corresponds  to  the  world  external  to  the  network.)  Suppose  that  the  routing  matrix  P 
is  such  that  it  is  possible  for  a  customer,  upon  entering  j  and  before  returning  to  it.  to 
follow  a  route  in  which  station  k  is  visited  precisely  />*  times  (note  that  we  must  necessarily 
have  h ,  =  l);  such  an  h  is  termed  a  j-accessible  visit  vector.  A  customer  who  follow? 
such  a  route  is  said  to  follow  h  upon  entering  j.  The  time  it  takes  a  specified  customer 
to  follow  h  is  called  its  sojourn  time  along  h.  For  a  j-accessible  h,  let  Dn{j,h,t )  be  me 
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sojourn  time  (in  the  n’th  approximating  network)  along  h  of  the  first  customer  that  follows 
h  upon  entering  j  after  time  <. 

Before  stating  the  limit  theorem  for  Dn(j.h, t),  recall  that  Theorem  16  implies  that  the 
queue- lengths  at  each  of  the  d  stations  (in  the  n’th  approximating  network)  is  of  order 
nl/I  in  magnitude.  Since  each  station  has  first  come/first  serve  queueing  discipline,  it  is 
evident  that  the  amount  of  time  that  a  customer  spends  in  the  queue  of  station  ;  is  roughly 
Qj/Hj ,  where  Q,  is  the  queue-length  at  the  instant  at  which  the  customer  joins  the  queue 
at  station  j.  (We  obtain  the  approximation  Qj/m  by  noting  that  customers  at  station  j 
are  served  (asymptotically)  at  rate  n,.)  Hence,  the  sojourn  time  of  a  customer  in  the  n  th 
system  is  of  order  n1/J  in  magnitude.  Since  time  is  measured  in  units  of  order  n  in  the 
n’th  network  (see  Theorem  16),  this  implies  that  a  customer  visits  all  the  stations  along  its 
route  instantaneously  in  the  diffusion  limit.  Hence,  in  the  diffusion  time  scale,  queues  and 
workloads  seen  by  a  customer  arriving  to  station  j  freeze  while  the  customer  follows  h.  As 
a  consequence,  the  limit  process  for  sojourn  times  has  the  property  that  the  order  in  which 
stations  sure  visited  along  a  route  does  not  affect  the  diffusion  limit  of  the  route’s  travel 
time.  Furthermore,  the  diffusion  limits  of  Dn(j,h,t)  and  D„(k,h,t)  are  identical,  provided 
that  h  is  both  j-accessible  and  i-accessible:  see  REIMAN  (1984)  for  details. 

Theorem  20.  Assume  (7.3)  and  (7.12).  Let  (?(<)  =  . . .  ,<?*(<))  be  the  limit  process 

specified  by  Theorem  16.  Set  Dn(j,h,t)  =  Dn(j, h,nt)/nlf7.  Then, 

* 

in  D*[0,oo),  as  n  —  oo. 

We  conclude  this  section  with  a  brief  discussion  of  some  extensions  of  the  open  queueing 
model  that  has  been  described  above.  Firstly,  (7.3)  can  be  relaxed  to  deal  with  networks 
in  which  certain  stations  are  not  asymptotically  in  heavy  traffic.  (In  other  words,  there 
exists  at  least  one  station  i  for  which  #  (>(/  —  P)-1 ). •)  This  generalization  has  been 
fully  explored  in  CHEN  and  MANDELBAUM  (1988).  Basically,  the  stations  of  such  a 
network  can  be  classified  into  one  of  three  types:  balanced,  non- bottleneck,  and  strict 
bottleneck  stations.  At  a  balanced  station,  the  “effective  arrival  rate’  (both  internal  and 
external  arrivals)  is  roughly  identical  to  the  service  rate,  and  the  station  is  in  heavy  traffic. 
On  the  other  hand,  at  a  non-bottleneck  station,  the  effective  arrival  rate  is  less  than  tne 
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service  rate,  so  that  the  station  is  in  light  traffic.  As  a  consequence,  the  queue-length  at 
a  non-bottleneck  station  is  basically  bounded  (in  a  stochastic  sense)  as  a  function  of  n. 
Thus,  in  the  diffusion  re-scaling  in  which  the  spatial  variables  are  measured  in  units  of 
n1/J,  the  non-bottlenecks  are  essentially  drained  of  customers  instantaneously,  after  which 
the  corresponding  queue-length  processes  are  effectively  zero.  Hence,  the  diffusion  limit 
gives  no  information  about  the  queue-lengths  at  the  non-bottleneck  stations.  In  some 
sense,  the  queueing  network  can  then  be  reduced  to  an  analysis  of  the  remaining  balanced 
and  strict  bottleneck  stations.  This  is  an  example  of  what  is  known,  in  the  literature, 
as  "state  space  collapse.”  This  should  not  be  interpreted  as  suggesting  that  the  non¬ 
bottlenecks  play  no  role  in  the  diffusion  limit  for,  in  fact,  they  do.  For  example,  the 
diffusion  limit  needs  to  keep  track  of  how  customers  move  between  stations  in  the  reduced 
network  consisting  of  balanced  and  strict  bottleneck  stations.  The  routing  matrix  of  the 
full  network  (including  non-bottleneck  stations)  must  be  analyzed  in  order  to  calculate  the 
submatrix  which  characterizes  flows  within  the  reduced  network. 

The  behavior  of  the  network  at  strict  bottlenecks  is  comparable  to  that  of  a  queue 
in  which  the  arrival  rate  is  strictly  greater  than  the  service  rate.  As  we  saw  in  Section 
5.  oversaturated  queues  have  very  simple  behavior.  In  particular,  the  queue-lengths  grow 
linearly  in  time.  When  appropriately  centered  to  reflect  the  linear  trend,  the  queue-length 
process  exhibits  fluctuations  of  order  n1/2.  These  fluctuations  are  contributed  by  a  non¬ 
reflecting  Brownian  motion  term  (from  the  non-bottleneck  stations)  as  well  as  a  reflecting 
Brownian  motion  term  (to  handle  input  from  the  balanced  stations). 

Thus,  the  components  of  the  vector-valued  diffusion  limit  display  three  different  types 
of  behavior.  Components  corresponding  to  non- bottlenecks  vanish  identically,  whereas 
balanced  components  exhibit  reflection  at  the  boundary  of  the  non-negative  orthant.  On 
the  other  hand,  the  strict  bottlenecks  display  no  non-trivial  boundary  behavior  (other 
than  that  induced  from  the  balanced  stations).  This  is  because  the  diffusion  limit  merely 
characterizes  the  non-zero  fluctuations  about  the  linear  drift  term,  as  the  queue-lengths 
drift  to  infinity.  These  fluctuations  are  of  arbitrary  sign. 

We  turn  now  to  discussion  of  a  second  generalization  of  the  queueing  networks  con¬ 
sidered  in  this  section.  Suppose  that  the  network  serves  two  types  of  customers,  one  type 
of  which  has  a  pre-emptive  resume  priority  over  the  other  at  each  station  of  the  network 
In  this  setting,  the  high  priority  customers  are  unaffected  by  the  low  priority  customers 
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Hence,  in  heavy  traffic,  the  high  priority  customers  race  through  the  network  relative  to  the 
low  priority  customers.  Thus,  the  queue-length  processes  (at  the  various  stations)  of  the 
high  priority  customers  are  negligible  compared  to  those  of  the  low  priority  customers.  As 
a  consequence,  in  the  diffusion  limit,  the  components  corresponding  to  high  priority  cus¬ 
tomers  vanish  identically.  However,  the  effect  of  the  high  priority  customers  is  manifested 
in  the  limit  processes  obtained  for  the  low  priority  customers;  see  JOHNSON  (1983)  for 
further  details.  It  is  also  possible  to  obtain  diffusion  approximations  for  multiple  customer 
type  networks,  in  which  the  routing  is  “feedforward”  (i.e.,  the  stations  can  be  numbered 
so  that  the  route  of  each  customer  type  is  an  increasing  sequence).  The  queue  discipline 
at  each  station  j  is  defined  by  a  partition  of  the  customer  types  into  subsets  H,  and  L,. 
Customers  of  type  icff,  have  pre-emptive  resume  priority  over  those  of  type  kcLr  Within 
each  subset  H ,  and  L:,  the  queue  discipline  is  first  come/first  served.  As  in  the  previous 
priority  queue  that  was  described,  the  queue-lengths  of  the  high  priority  customers  at 
a  particular  station  vanish  in  the  diffusion  limit.  On  the  other  hand,  the  queue-lengths 
corresponding  to  lower  priority  customers  at  a  particular  station  are  represented  by  fixed 
multiples  of  a  single  limiting  marginal  process  corresponding  to  total  queue-length  at  that 
station.  Thus,  the  diffusion  limit  for  such  a  d  station  queueing  network  (with  multiple 
customer  types)  is  basically  a  d  dimensional  process.  This  is  another  example  of  “state 
space  collapse”  (i.e.,  the  diffusion  limit  is  d- dimensional,  whereas  the  exact  model  records 
the  number  of  customers  of  each  type  at  each  station  and  is  therefore  higher  dimensional  ): 
this  limit  theorem  is  described  in  PETERSON  (1985). 

For  single  station  open  systems,  several  other  diffusion  approximations  have  been  stud¬ 
ied.  JOHNSON  (1983)  considers  a  single  station  model  with  two  or  more  customer  types 
under  first  come/first  serve  and  processor  sharing  queue  disciplines.  A  heavy  traffic  dif¬ 
fusion  limit  is  obtained  for  the  total  queue-length  process;  it  is  then  shown  that  a  fixed 
fraction  of  the  limit  process  corresponds  to  customers  of  a  particular  type.  A  similar  result 
is  obtained  in  REIMAN  (1983).  The  heavy  traffic  behavior  of  a  two  queue  system  in  which 
customers  join  the  shortest  queue  is  shown  to  converge  to  a  limit  process  with  equal  frac¬ 
tions  of  customers  at  each  of  the  two  stations.  Thus,  in  the  limit,  only  the  total  number  of 
customers  in  the  system  varies  stochastically.  A  further  example  of  “state  space  collapse 
appears  in  REIMAN  (1988),  where  a  multi-class  feedback  queue  with  round-robin  service 
discipline  is  studied. 
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We  conclude  this  discussion  of  generalizations  of  Theorem  16  by  pointing  out  that 
the  i.i.d.  assumptions  on  the  inter-arrival  times,  service  times,  and  routing  vectors  are 
unnecessary  to  the  basic  argument.  As  suggested  by  (7.4)-(7.6),  all  that  is  really  needed 
is  that  the  various  input  proceses  satisfy  a  joint  FCLT.  This  point  of  view  is  stressed  in 
the  modeling  approach  described  i:i  HARRISON  and  WILLIAMS  (1987)  and  HARRISON 
(1985). 


8.  Diffusion  Approximations  for  a  Closed  Network  of  Queues  in  Heavy  Traffic 

In  this  section,  we  describe  the  closed  network  analogs  of  the  results  given  in  Section 
7  for  open  networks.  We  start  by  describing  the  basic  model.  As  in  the  open  case,  we 
consider  a  network  of  d  queueing  stations,  in  which  each  station  has  an  infinite  waiting 
room  and  a  single  work-conserving  server  that  serves  customers  in  the  order  in  which  they 
arrive.  The  customer  routing  between  stations  is  assumed  to  be  Markovian  and  is  described 
by  a  stochastic  matrix  P  (called  the  routing  matrix);  we  require  that  P  be  irreducible  and 
that  Pa  =  0  for  1  <  i  <  d.  As  in  Section  7,  let  V,  =  {K(m) :  m  >  1}  be  a  sequence  of  i.i.d.  unit 
mean  r.v.’s  for  which  6?  *  varVi(m)  <  oo.  If  >*>( n)  is  the  service  rate  at  the  j’th  station  in 
the  n’th  network,  then  V}(n)/p j(n)  is  the  actual  duration  of  service  for  the  m’th  customer 
served  at  the  j’th  station  in  the  n’th  network. 

We  assume  that  the  n’th  approximating  network  contains  precisely  |n1/2J  customers. 
Thus,  the  parameter  n  that  indexes  the  system  here  has  a  physically  tangible  meaning 
(unlike  the  open  case);  it  characterizes  the  total  number  of  customers  in  the  closed  network. 
Let  g„(0)  =  (<5nl(0),.  ..,<?„*(0))  be  a  random  d-vector  having  non-negative  integer-valued 
components  for  which  <Jnl(0)  +  •  ■  •  +  Qn*(0)  =  [nI/2J.  The  j’th  component  is  to  be  interpreted 
as  the  number  of  customers  that  are  sitting  in  the  queue  of  the  j’th  station  waiting  to 
be  served  at  time  t  =s  0  (in  the  n’th  system).  We  require  that  V),...,  Vrf,Q„(0)  be  mutually 
independent.  The  network  that  we  have  just  described  is  termed  a  generalized  closed 
Jackson  network. 

To  obtain  heavy  traffic  behavior  ar  °ach  of  the  d  stations,  we  require  that  the  service 
rate  v,{n)  be  approximately  proportional  to  where  t  =  (*t,  ■  ,*•*)  is  the  unique  stationary 

probability  vector  associated  with  P  (i.e..  tp  =  *•).  Thus,  heavy  traffic  is  obtained  when 
the  service  rate  is  roughly  proportional  to  the  relative  expected  number  of  visits  that  a 
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customer  makes  to  a  given  station.  Stated  in  mathematical  terms,  we  require  that 

(8.1)  nl/*(M(f»)  -  n)  -  c 

as  n  —  oo,  where  rt/m  =  d  (a  constant).  Since  the  diffusion  approximation  will  be  obtained 
by  speeding  up  time  by  a  factor  of  n  and  re-scaling  space  by  a  factor  of  nl/J,  we  can  re¬ 
express  the  conditions  for  heavy  traffic  in  the  following  way.  A  diffusion  limit  will  be  a 
reasonable  approximation  to  a  closed  queue  if  the  total  number  of  customers  m  is  large 
and  the  service  rate  p,  at  the  i’th  station  has  the  property  that 

/  ».  \  (  *i\ 

max  —  -  min  — 
i<*<^  \m.  y 

is  roughly  of  order  1/m.  Then,  the  diffusion  limit  will  describe  fluctuations,  in  the  queueing 
network,  of  order  m  on  the  time  scale  of  order  mJ. 

As  in  the  open  network  setting,  the  theory  of  diffusion  approximation  for  closed  systems 

depends  critically  on  representing  the  vector  queue-length  process  Qn(t)  =  (Qnl(t) . Qn«(t)) 

as  a  regulated  version  of  the  centered  input  processes  corresponding  to  the  service  times 
and  routing  vectors.  Specifically,  one  represents  Qn  as  Q„  =  f(Z„),  where  Zn  is  a  process 
that  can  be  approximated  by  a  Brownian  motion  and  f[z)  =  r  +  h(z)(l  -  P),  where  h  satisfies 
(7-9). 

To  state  the  diffusion  approximation  for  the  above  class  of  closed  networks,  we  need 
to  further  assume  that  there  exists  a  random  d- vector  Q(O)  such  that 

(8  2)  n'l/,gn(0)  =>  g(0) 

as  n  —  oo;  the  vector  Q(0)  must  necessarily  have  components  that  add  to  one.  Then,  the 
process  Z„  (when  appropriately  re-scaled)  can  be  approximated  by  Z,  where 

(8-3)  Z(t)  =  Q( 0)  +  c(/  -  P)t  +  rl/aS(t). 

the  random  elements  Q{ 0)  and  B  appearing  in  (8.3)  are  independent.  Also,  B  is  a  standard 
Brownian  motion,  c  is  the  vector  appearing  in  (8.1)  and  r  is  the  covariance  matrix  given 
by 

d 

f;*  =  1  +  &,3)  ~  Pjk  “  HkbkPk)  ~  ^  VlPtjPlki  1  -  &?)• 

'=* 

By  applying  continuous  mapping  ideas  to  the  mapping  /,  one  obtains  the  following  result 
see  CHEN  and  MANDELBAUM  (1988)  for  a  complete  proof. 
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Theorem  21.  Assume  (8.1)  and  (8.2).  Let  Q"(t)  =  Q„(nt)/nl/ J.  Then, 

Qn  =>  f(Z)  =  Q 

in  D««(0,oo)  as  n  —  oo,  where  Z  is  defined  by  (8.3). 

We  observe  that  since  the  vector  Qn  is  normalized  by  n1/J  to  obtain  Qn,  it  is  evident  that 
Q  must  be  a  stochastic  process  that  lives  on  the  simplex  S  =  {zctt* :  *,  >  0.  *i+  =  1}.  As 

a  consequence,  the  covariance  matrix  r  must  be  singular  (since  the  (/-dimensional  Brownian 
motion  Z  must  lie  in  a  (d-  1)  dimensional  subspace).  We  henceforth  assume  that  r  is  such 
that  the  (d-  1)  x  (d~  1)  principal  submatrices  of  r  are  positive  definite.  Because  of  property 
(7.10)  of  the  regulator  map.  it  turns  out  that  Q  is  (as  in  Section  7)  a  diffusion  process.  In 
fact,  Q  can  be  viewed  as  a  regulated  version  of  the  Brownian  motion  Z.  The  regulation 
forces  Z  to  “reflect’’  in  the  direction  of  the  i’th  row  of  I  -  P  whenever  the  i’th  component 
of  Q  is  zero. 

Since  Q  takes  values  in  the  component  set  S ,  it  seems  reasonable  to  expect  that  Q 
possesses  a  stationary  distribution  *.  This  is  the  principal  content  of  the  following  result, 
due  to  HARRISON,  WILLIAMS,  and  CHEN  (1989). 

Theorem  22.  The  diffusion  process  Q  has  a  unique  stationary  probability  distribution  n. 
The  probability  distribution  n  has  a  strictly  positive  density  p(z)  with  respect  to  Lebesgue 
measure  on  5. 

Since  a  certain  subclass  of  the  diffusion  processes  Q  described  above  can  be  obtained  as 
limits  of  Markovian  Jackson  networks  having  “product  form”  stationary  distributions,  one 
hopes  that  the  product  form  theory  carries  over  to  the  diffusion  setting,  thereby  permitting 
p(r)  to  be  calculated  explicitly  in  certain  cases.  We  say  that  p  is  an  exponential  density 
if  it  can  be  representd  in  the  form 

4 

p(*)  =  c  J^exp(-rhK) 

IS  1 

for  zcS  (for  some  constant  C).  The  next  result  is  also  due  to  HARRISON,  WILLIAMS, 
and  CHEN  (1989). 

Theorem  23.  The  density  p  of  the  stationary  distribution  t  is  an  exponential  density  it 
and  only  if 

(8  4)  2T;t  =  -(Pkj^kk  +  PjkTkk) 
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for  j  ^  k.  Furthermore,  if  p  is  an  exponential  density,  then 

d 

P(*)  =  C 

l 

where 

=  IhcJTh, 

C  -  f  JIexp(-f**,)m(<ix) 

Js  1=1 

and  m  is  Lebesgue  measure  on  5. 

As  in  the  open  case,  condition  (8.4)  is  automatically  satisfied  when  6?  =  1  (i.e..  the 
service  times  have  the  same  coefficient  of  variation  as  does  an  exponential  r.v.).  Also,  if 

(8.4)  is  satisfied  and  /i(n)  =  p  for  n  >  1,  then  c  =  0  and  the  stationary  distribution  t  then 
evidently  reduces  to  uniform  distribution  on  the  simplex. 

We  conclude  our  discussion  of  the  qualitative  structure  of  the  process  Q  by  describing 
the  analog  of  the  basic  adjoint  relationship  (7.16)  for  closed  networks;  see  HARRISON. 
WILLIAMS,  and  CHEN  (1989)  for  further  details. 

Theorem  24.  Let  be  defined  as  in  Section  7.  Suppose  *  is  the  stationary 

probability  distribution  for  Q.  Then,  there  exist  finite  measures  i/t, . . . ,  v*  on  Fi  nS . Fi  r  5 

such  that  for  each  fcC ’, 

(8.5)  /  (Lf)(z)p{z)m(dx)  +  [  (D, /)(*)«(<**)  =  0. 

Js  i  Jr.ns 

The  development  of  numerical  solvers  for  dealing  with  (8.5)  remains  an  important  open 
problem. 

Diffusion  limits  can  also  be  obtained  for  the  sojourn  times  that  were  defined  in  Section 
7.  The  basic  limit  theorem  is  identical  to  Theorem  20  and  the  statement  is  omitted. 

As  for  generalizations  of  the  closed  model  that  has  been  described  here,  a  number  of 
possibilities  have  been  investigated  in  the  literature.  In  HARRISON,  WILLIAMS,  and 
CHEN  (1989),  the  assumption  that  the  service  requirements  at  each  section  are  i.i.d.  is 
dropped,  and  replaced  with  a  requirement  that  the  input  processes  satisfy  funct  onal  central 
limit  theorems.  Although  no  proof  is  offered,  the  paper  does  calculate  the  appropriate 
diffusion  limit  for  such  a  model.  Closed  networks  in  which  (8.1)  is  weakened  to  permit  the 
possibility  of  including  stations  in  light  traffic  are  studied  in  CHEN  and  MANDELBAUM 


(1988).  As  in  the  open  case,  the  queue-length  populations  at  the  light  traffic  stations 
vanish  in  the  diffusion  limit.  In  addition,  closed  networks  with  multiple  customer  classes 
are  considered  in  CHEN  and  MANDELBAUM  (1988).  The  priority  ranking  of  the  various 
customer  classes  is  assumed  to  be  the  same  at  each  station  of  the  network.  Again,  the 
theory  is  similar  to  that  obtained  in  the  open  case;  high  priority  customers  disappear  in  the 
diffusion  limit,  although  they  do  influence  the  structure  of  the  limiting  process  associated 
with  the  lower  priority  customers. 

9.  Approximations  for  Queues  with  Many  Server* 

In  this  section,  we  discuss  limit  theorems  for  queues  in  which  each  station  possesses 
a  single  waiting  room  and  a  large  number  of  servers.  Customers  are  assigned  to  the 
first  available  server  on  a  first  come/first  serve  basis.  The  types  of  limits  obtained  here 
will  typically  not  exhibit  any  of  the  boundary  behavior  that  characterized  the  reflecting 
Brownian  motions  studied  in  the  heavy  traffic  settings  of  Sections  5,  7,  and  8.  On  the 
other  hand,  the  limit  processes  that  arise  here  need  not  be  Markov  processes.  Thus,  the 
approximations  typical  of  the  many  server  context  are  not  true  diffusion  approximations, 
since  the  limit  processes  need  not  be  diffusions.  However,  we  choose  to  discuss  these 
approximations  here  because  of  their  intrinsic  importance  and  because  the  ideas  required  to 
derive  these  limits  are  largely  identical  to  those  used  to  obtain  the  diffusion  approximations 
described  earlier  in  this  chapter  (see  Sections  2  and  3). 

We  initiate  this  discussion  by  considering  a  single  station  queue  with  an  infinite  number 
of  servers.  Interesting  limit  behavior  is  obtained  by  sending  the  queue  into  heavy  traffic. 
Heavy  traffic,  in  this  setting,  means  that  the  arrival  rate  is  high,  so  that  the  expected 
number  of  busy  servers  is  large.  More  precisely,  consider  a  sequence  of  GlfC/oo/oo  queues 
constructed  in  the  following  manner.  The  service  time  sequence  V  =  {K  :  «  >  1}  is  i-i-d. 
with  common  distribution  F,  whereas  the  inter-arrival  times  in  the  n’th  system  are  an 
independent  sequence  Un  =  {Uni  i  >  1}  of  i.i.d.r.v.'s  in  which  Un,  can  be  represented  as 
l\/n\  we  assume  that  the  system  is  idle  at  t  =  0,  for  simplicity.  Note  that  in  the  n  th 
system,  the  inter-arrival  times  are  re-scaled  so  that  arrivals  are  occurring  n  times  faster 
than  in  the  first  system,  whereas  the  service  times  are  not  re-scaled.  This  is  in  contras: 
to  the  diffusion  approximations  previously  discussed  in  this  chapter,  in  which  both  inter- 
arrival  times  and  service  times  are  re-scaled  simultaneously.  In  any  case,  we  will  estabiisr. 
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a  limit  theorem  for  the  queue-length  process  at  the  station  as  the  parameter  n  tends  to 
infinity. 

The  result  is  most  transparent  when  the  service  time  r.v.’s  have  a  discrete  distribution 
with  finite  support.  Suppose,  in  particular,  that  F  assigns  probability  pi  to  the  value  x,. 
for  1  <  »'  <  m.  Let  /MO  be  the  total  number  of  customers  having  received  service  time  x, 
by  time  t  in  the  n’th  system.  Because  the  n’th  inter-arrival  stream  is  obtained  from  the 
first  inter-arrival  stream  by  speeding  it  up  by  a  factor  of  n,  we  may  write  /MO  =  N,(nt j. 
where  N,(  )  =  /Vtl( ).  Let  /V(t)  =  Ny(t)+  ■  ■  +  jVm(0  be  the  total  number  of  customers  to  arrive 
by  time  t  in  the  first  system.  Then, 

*(<) 

•v.(0  =  £  l{V}  st|). 

;  =  i 

Suppose  that  a 2  =  varC'i  <  ^  and  that  A-1  =  Ei\  >  0.  Then,  the  multivariate  version  of 
Donsker’s  theorem  applies  (see  Theorem  4),  yielding  the  fact  that 


(9  1) 


/ ,  l"'/  ,  Ln,J  ,  L"‘J  \ 

"1/J  . =xm)-tpm.-53c/,-tA-> 

\  i=i  n  i  =  i  i=i  / 


converges  to  a  Brownian  motion  taking  values  in  Rm+l.  Using  a  random  time  change 
argument  similar  to  that  used  to  obtain  (5.1),  we  can  substitute  t'  =  N(nt)/n  into  (9.1). 
thereby  yielding  the  fact  that 

_1/1  -v(n0_  .Vm(nO  iV(nt)_  4  ,V(nOx_!\ 

Tt  1  — —  —  Pl i  ‘  "  —  Pm  1 1  —  ———A  1 

\  n  n  n  n  n  / 

converges  to  a  0T"+l- valued  Brownian  motion.  The  continuous  mapping  principle  then 
implies  that 


(9.2) 


(Saffi-j*. . *=!2- w) 


converges  to  an  m-dimensional  Brownian  motion  B  -  (B(t)  =  ( (<), . . . ,  Bm(())  t>  0).  Re¬ 
calling  that  the  service  times  for  customers  corresponding  to  Nn,{t)  are  all  identical  to  x,. 
we  find  that  Qn,(t)  =  .MO  -  /M(<  -  x,)  AO),  where  Qn,{t)  is  the  number  of  customers  at 
the  station  at  time  t  in  the  n’th  system  that  were  assigned  service  time  x,.  Consequently 
Qn(t)  (the  total  number  of  customers  at  the  station  at  time  t  in  the  n’th  system)  can  be 
represented  as 

m 

<?„«)  =  53  <• mo-m(<-x.)ao)). 
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It  then  follows  from  (9.2)  that 

(9.3)  n =>f]  (£?,(<) -fl.((t-r,)V  0)) . 

'  '  >*i 

Since  Brownian  motion  has  finite  dimensional  distributions  that  are  Gaussian,  it  is  ev¬ 
ident  that  the  same  must  be  true  of  the  limit  process  appearing  in  (9.3).  As  a  consequence, 
the  limit  appearing  in  (9.3)  is  termed  a  Gaussian  approximation.  Gaussian  processes 
are  highly  tractable,  since  their  finite  dimensional  distributions  are  totally  characterized 
by  their  mean  and  covanance  functions.  Thus,  these  limit  processes  are  somewhat  easier 
to  use  (when  applicable)  than  the  diffusion  limits  of  Sections  7  and  8,  since  the  diffusions 
obtained  there  typically  require  sophisticated  numerical  routines  to  calculate  performance 
measures  of  interest.  We  also  note  that  (9.3)’s  limit  process  has  the  interesting  property 
that  for  t  >  max{x,  :  1  <  i  <  m},  the  marginal  distribution  is  independent  of  t  (i.e.,  the  system 
reaches  steady-state  in  finite  time). 

A  straightforward  calculation  shows  that  the  covariance  function  of  the  limit  appearing 
in  (9.3)  is  given  by 

(9  4)  c(a,s  +  t)=\[  F(u)(l  -  F(t  + u))du  +  <rJA3  /  (1 -/*(!  + u))(l  -  F(ti))du 

Jo  Jo 

for  s.t  >  0  (note  that  the  mean  of  (9.3)’s  limit  is  identically  zero).  Since  r.v.’s  with  arbitrary 
distribution  can  be  approximated  by  discrete  r.v.’s,  this  suggests  that  the  above  limit 
theorem  ought  to  hold  more  generally.  The  following  theorem  is  due  to  BOROVKOV 
(1967). 

Theorem  25.  Consider  a  sequence  of  GI/G/o c/oc  queues  constructed  as  described  earlier 
in  this  section.  Suppose  that  A-1  =  EU1  >  0  and  a 3  =  v*tUx  >  oo.  If  varVi  <  oc,  then 

=><?(<) 

in  Dr<(0.3c),  where  {Q{t)  :  t  >  0}  is  a  process  having  Gaussian  finite  dimensional  distribu¬ 
tions.  Furthermore,  EQ(t)  =  0  and  its  covariance  function  is  given  by  (9.4). 

In  contrast  to  the  diffusion  limits  obtained  earlier  in  this  chapter,  the  distribution  of 
the  Gaussian  limit  Q  =  {Q(t)  :  t  >  0}  depends  on  the  entire  service  time  distribution  F .  no* 
just  on  its  mean  and  variance.  Thus,  the  tail  behavior  of  the  sendee  times  has  a  significant 
impact  on  an  infinite  senet  queue  in  heavy  traffic.  We  further  note  that  if  Fit)  - 
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then  the  covariance  function  is  identical  to  that  of  an  Omstein-Uhlenbeck  process.  As  a 
consequence,  it  turns  out  that  for  a  GI/M/oo/oo  queue,  the  limit  process  Q  is  an  Omstein- 
Uhlenbeck  process  with  infinitesimal  mean  -p*  end  infinitesimal  variance  (A3<rJ  +  A).  Thus, 
in  the  case  of  exponential  service  times,  the  limit  is  a  diffusion.  However,  Q  is  typically  not 
Markov.  GLYNN  ( 1982)  shows  that  Q  is  Markov  if  and  only  if  F(t)  =  1  -  pe-"*  for  0  <  p  <  l 
and  p  >  0. 

The  process  <J(t)  converges  in  distribution  to  a  limit  Q(ca)  as  t  — « oo.  The  steady-state 
r.v.  Q(oo)  is  normally  distributed  with  zero  mean  and  variance 

*  =  £  (l  +  (A V  -  l)p^°°(l  -  , 

where  p"1  =  £Vi.  This  is  another  confirmation  of  the  analytical  tractability  of  the  Gaussian 
limit  <?;  clearly,  Q(oc)  can  be  used  as  an  approximation  to  the  long-run  behavior  of  a 
Gl/Gj oc/oo  queue  in  heavy  traffic. 

Similar  Gaussian  approximations  to  Theorem  25  can  be  obtained  for  the  cumulative 
departure  process  of  a  GI/G/ oo/oo  queue  of  the  type  described  above;  see  WHITT  ( 1984)  for 
further  details.  In  addition,  Gaussian  limits  can  be  derived  for  networks  of  infinite  server 
stations;  the  relevant  techniques  are  sketched  out  in  WHITT  (1982).  Finally,  it  turns  out 
that  one  can  extend  these  results  to  finite  server  stations  in  which  the  number  of  servers 
increases  with  the  arrival  rate  suitably  rapidly.  Note  that  the  number  of  busy  servers 
(according  to  Theorem  25)  in  the  n’th  infinite  server  queue  is  approximately  n\EVx.  Hence, 
if  the  number  of  available  servers  sn  associated  with  the  n’th  system  grows  sufficiently  more 
rapidly  than  nXEVu  the  finite  server  model  will  act  asymptotically  like  the  infinite  server 
system.  Specifically,  this  holds  if  nl/,( tn  -  n\EVx)  —  oo  as  n  —  oo. 

We  conclude  this  section  by  briefly  discussing  approximations  for  closed  networks  with  a 
large  number  of  servers  at  each  station.  Again,  we  start  by  considering  the  case  where  each 
of  the  d  stations  has  an  infinite  number  of  servers.  We  assume,  for  concreteness,  that  the 
customer  routing  between  stations  is  Markovian  (although  this  cam  easily  be  generalized ). 
In  addition,  the  service  time  streams  for  each  of  the  d  stations  form  independent  sequences 
of  i.i.d.  random  variables  with  continuous  distributions.  We  further  assume  that  at  t  =  0. 
all  the  customers  in  the  network  are  sitting  at  the  first  station,  waiting  to  be  served. 
We  shall  describe  a  limit  theorem  for  the  network  by  letting  the  number  of  customers  n 
contained  within  the  network  tend  to  infinity.  In  contrast  with  previous  approximations 
that  we  have  analyzed,  there  will  be  no  need  to  re-scale  time  in  any  way. 
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The  key  observation  here  is  to  recognize  that  since  each  station  has  an  infinite  number 
of  servers,  customers  never  queue  for  service.  As  a  consequence,  customers  do  not  interfere 
with  each  other  as  they  circulate  through  the  network.  Hence,  each  of  the  n  paths  followed 
by  customers  through  the  network  are  independent.  Furthermore,  it  is  clear  that  these 
paths  are  identically  distributed.  Let  A,(t)  be  the  station  occupied  by  the  i’th  customer 
at  time  t.  Then,  Qn}(t),  the  number  of  customers  at  the  j’th  station  at  time  t  in  the  n’th 
system,  is  given  by 

ft 

gnj(<)  =  £/(*(<)  =  ;)• 

■  si 

If  we  let  Y,(t)  =  (I(X,(t)  =  1) _ 7(,Y.(0  =  d)),  we  conclude  that  <?„(<)  =  (<3m(0 . Qm{t))  can 

be  expressed  as 

n 

<?n(0  =  S>.(0 

»  =  i 

and  hence  Qn  =  { Qn(t )  :  t  >  0}  can  be  expressed  as  a  sum  of  n  i.i.d.  £>B*[0,c»)- valued  random 
elements.  Centred  limit  theorems  exist  for  such  objects.  When  applied  in  this  setting,  we 
obtain  the  following  limit  theorem;  see  GLYNN  and  KURTZ  (1989)  for  additional  details. 

Theorem  20.  Let  Q  be  the  Gaussian  process  with  covariance  function  identical  to  that 
of  Yx.  Then, 

-£yl(o)  =><?(*) 

* 

as  n  —  so,  in  £)«<[(), oo). 

The  limit  process  Q  -  {Q(t)  •  t  >  0}  is  again  a  Gaussian  process  that  is  typically  non- 
Markovian.  Since  the  processes  Qn  are  Markov  when  the  service  times  are  exponential,  the 
same  property  is  inherited  by  the  limit  Q  in  that  case,  however. 

Suppose  now  that  the  routing  matrix  is  irreducible  and  that  at  least  one  station  has 
an  associated  service  time  distribution  that  is  spread-out  (i.e.,  some  n-fold  convolution  of 
the  distribution  possesses  a  density  component).  We  further  require  that  the  mean  n~l  of 
the  service  time  distribution  for  the  Tth  station  is  finite  for  1  <  i  <  d.  Then,  <J„(0  =>  QJ  *  1 
at  t  —  x,  for  some  limiting  r.v.  Q„(oc).  The  following  result  gives  an  approximation  to 
Qf,(oc)  when  the  number  of  customers  n  in  the  network  is  large. 

Theorem  27.  Let  *  be  the  unique  stationary  distribution  of  the  routing  matrix  P  Se: 
P.  =  (Hj-i  t;P7’)  an<^  ^et  P~{P-  Pi'-  Then. 


„i/j  -  p)  r't'Nvu) 

in  R4,  when  r«  =-pi(l  -  Pi)  and  r,;  =  -p,p,  for  «  *  j. 

Limit  theorems  have  also  been  derived  for  closed  Jackson  networks  in  which  the  number 
of  servers  at  each  station  is  large  but  finite.  (Recall  that  in  a  Jackson  network,  service 
times  are  exponentially  distributed.)  The  limit  processes  in  this  setting  are  typically 
vector-valued  Omstein-Uhlenbeck  processes.  The  tools  that  are  used  here  are  somewhat 
different  from  those  described  earlier  in  this  chapter.  Rather  than  attempt  to  represent 
the  queue-length  process  as  some  continuous  functional  of  its  inputs  (thereby  permitting 
one  to  use  continuous  mapping  ideas),  the  technique  that  has  commonly  been  used  here 
is  to  show  that  the  infinitesimal  generator  of  an  appropriately  scaled  version  of  the  vector 
queue-length  process  converges  to  the  infinitesimal  generator  of  the  limiting  Omstein- 
Uhlenbeck  process.  This  approach  is  analytical,  in  contrast  to  the  more  probabilistic 
continuous  mapping  aproach  used  earlier  in  this  chapter.  For  two-station  networks,  the 
work  of  STONE  (1963)  on  weak  convergence  of  birth  death  processes  can  be  used.  For 
more  general  networks,  the  techniques  outlined  in  STROOCK  and  VARADHAN  (19791 
have  proved  successful.  For  further  details  on  these  limit  theorems,  see  IGLEHART  (19651 
and  PRISGROVE  (1987). 

10.  Conditional  Weak  Convergence  Theorems 

In  this  section,  we  briefly  describe  an  interesting  class  of  diffusion  limits  that  arise  as 
approximations  to  the  behavior  of  certain  stochastic  processes  when  conditioned  on  an 
appropriate  rare  event. 

Consider,  for  example,  the  behavior  of  the  waiting  time  sequence  {Wn  :  n  >  0}  of  the 
single  server  queue  (77/G/l/oo.  As  discussed  in  Section  5,  the  W7,’s  satisfy  the  recursion 
=  (W7, +*„+,]♦ ,  where  =  V„_i  -U„  and  V„.  Un  are  the  n’th  service  time  and  interarrival 
time,  respectively.  If  we  assume  that  the  zero’th  customer  encounters  an  idle  server,  then 
tV0  =  o.  We  shall  be  interested  in  the  behavior  of  the  waiting  time  sequence  within  the  first 
busy  period  (i.e.,  over  the  interval  (0.7'),  where  T  =  inf{n  >  1  :  Wn  =  0}).  We  note  that  if  we 
let  Sn  =  X,  {So  =  0),  Wn  =  Sn  for  n  <  7.  Hence,  we  can  alternatively  view  the  problem 
as  the  study  of  random  walk  over  the  interval  [0,7'),  where  T  -  mf{n  >  1  :  5„  <  0)  is  the 
time  of  first  entry  into  ( —  oc .  0] . 
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for  0  <  t  <  1.  We  note  that  XneDR[0, 1],  the  function  space  consisting  of  the  restrictions 
of  functions  in  £)j»(0,  oo)  to  the  interval  [0, 1}.  The  idea  is  now  to  study  the  behavior  of  X« 
within  the  first -busy  period.  One  way  to  accomplish  this  is  to  consider  the  conditional 
probability  distribution 

P«()  =  P{Xne  ■  \T  >  n}. 

This  permits  us  to  study  only  those  paths  of  the  waiting  time  sequence  in  which  the  first 
busy  period  is  still  in  progress  at  time  n.  The  following  theorem  is  due  to  IGLEHART 
(1974). 

Theorem  28.  Suppose  that  £|Xi|3  <  oc  and  that  X\  is  either  non-lattice  or  integer-valued 
with  span  1.  Then, 

Pn  =>  P 

as  n  —  oo  (in  Dr[ 0, 1]),  where  P( )  =  P{B+r}. 

Hence,  the  behavior  of  the  waiting  time  sequence,  when  conditioned  on  the  first  busy 
period  still  being  in  progress,  is  well  approximated  by  that  of  a  Brownian  meander. 

Brownian  excursion  also  arises  as  a  limit  of  the  waiting  time  sequence  when  conditioned 
on  the  behavior  of  the  first  busy  period.  Specifically,  let 

K(  )  =  P{XnC  \T  =  n}. 

Hence,  P'  describes  the  distribution  of  precisely  those  paths  of  the  waiting  time  sequence 
that  conclude  their  first  busy  period  at  time  n.  IGLEHART  (1975)  states  that  P'n  ^  P' 
as  n  —  oo,  under  certain  conditions  on  Xu  where  P'(-)  =  P(£^c}.  Note  that  Brownian 
excursion  returns  to  zero  at  time  t  =  1,  in  accordance  with  the  observation  that  the  n'th 
waiting  time  is  zero  if  T  =  n.  Further  conditioned  limit  theorems  of  the  above  type  may  be 
found  in  DURRETT  (1980),  KAIGH  (1976),  and  KAO  (1978). 

The  mathematical  tools  that  are  used  to  establish  conditioned  limit  theorems  are  some¬ 
what  different  from  those  described  earlier  in  this  chapter.  For  example,  even  if  the  un¬ 
conditional  approximating  processes  are  tight,  there  is  no  guarantee  that  the  conditioned 
approximations  will  be  tight  when  the  conditioning  event  has  probability  tending  to  zero. 
Hence,  the  problem  of  establishing  tightness  becomes  more  delicate  here.  As  a  conse¬ 
quence.  most  of  the  results  available  pertain  to  conditioned  limit  theorems  for  very  specific 
classes  of  stochastic  process  (for  example,  random  walk).  Typically,  the  structure  o:  the 
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process  plays  an  important  role  in  the  argument  that  is  needed.  However,  for  certain  types 
of  conditioned  limit  theorems,  general  tools  are  available;  see,  for  example,  DURRETT 
(1978). 
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